Axioms are important to get right, because all of mathematics rests on them. If there are too few axioms, you can prove very little and mathematics would not be very interesting. If there are too many axioms, you can prove almost anything, and mathematics would also not be interesting.

What is the purpose of axioms?

An axiom, postulate or assumption is a statement that is taken to be true, to serve as a premise or starting point for further reasoning and arguments.

Can math exist without axioms?

The bottom line is that the axioms are there, even if they are not mentioned explicitly. Yes, mathematics always needs Axioms. That is what mathematics is all about: Axioms and deductions. The difference between Axioms and Theorems is that Axioms require exactly zero deductions to be performed.