Either choose to leave the witch hunter to his demise or help him. If you leave the witch hunter in the hands of the bandits, he will be found laying dead with the three bandits looting his body in the same place you found him. If you help the witch hunter, you will have to fight the three level 14 bandits.

What happens if you don't help the Pellar?

If Geralt doesn't help the pellar:

He thus did not lift a finger as the witch hunters scattered the peasants and brought the ritual to a premature conclusion. As you can imagine, the pellar was not pleased by Geralt's indifference in the face of the witch hunters' persecution.

What happens if I give the crystal to Radovid?

Upon getting outside of the cave after picking the crystal, if you accept to give it away to the witch hunters (which arguably smells from the beginning), they will lie to Radovid and play the witcher for a fool.