Prefer and would rather can be used interchangeably. As you indicate, Omar, when we are talking about general preferences, prefer is followed by verb-ing, thus: I prefer listening to music to watching TV. I'd rather listen to music than watch TV.

Is rather prefer correct?

You can say "I'd rather you left" and "I'd prefer that you left" to mean the same thing, but "I'd rather prefer that you left" means something like "I would somewhat prefer that you left".

Can we use rather than with prefer?

We use “rather than” to compare 2 things that we would prefer to do: I'd prefer to walk than get a taxi. I'd prefer to walk rather than get a taxi.