But another sense of the word “hence” (“therefore”) causes more trouble because writers often add “why” to it: “I got tired of mowing the lawn, hence why I bought the goat.” “Hence” and “why” serve the same function in a sentence like this; use just one or the other, not both: “hence I bought the goat” or “that's why I ...

Is it wrong to say hence why?

It's incorrect because the word "why" was used as well, also the first part is screwed up. I haven't used them for years; hence I am selling them. You could also use "thus". The most natural word choice would be "so".

Why do we say hence why?

The hence in both case means therefore and it is like saying therefore, why would anyone value. The why in both cases is the beginning of the next phrase and incidental.