The other side of the coin is tails, tails are on the down side and tails is upside down to heads. Australia is like tails on a coin with heads up, OK. Australia is located on the downside, making us upside down to the people on the top side of a place coined 'Earth'.
Do you feel upside down in Australia?
Remember that the Earth is a sphere, like a giant ball: so there is no "up" or "down", since a sphere is symmetric. That is, it looks the same no matter what way you look at it. So, people in Australia have just as much right to call themselves "up" as people in the Northern Hemisphere do!
Why is Australia depicted upside down?
It looks upside down because now your feet are pointed in the same direction (generally) as people in the southern hemisphere, and it's backwards because YOU reversed left-and-right. "But I don't FEEL upside-down when I go to Australia." The reason is - you have a brain.