It's even worse than that, because rationing makes driving an unreliable solution at any price point. Sure, a rich person can afford a car they only use part of the time. But if they regularly have a need to go into the city, they still need an alternative solution the rest of the time when they can't drive into the city for any amount of money. And having the ability to drive only some of the time is even more unhelpful for non-rich folks for exactly the reason you outlined.

The idea of rich people commuting by driving into Manhattan from their mansions in Connecticut while average people live closer and take public transit might bother some egalitarian sensibilities, but it's probably a better outcome than driving being so unavailable to everyone that the only people who really benefit from it are tourists or those who otherwise don't really need to drive into the city at all.