Because voice recognition is horribly imprecise. If you're controlling essential functions for driving then you need controls that are efficient, predictable, and reliable.
Sounds like a implementation problem, not a problem with voice control.
We have a 20 old navi with voice control. You can't just say free form things, but it's very deterministic. Most commands you want to say aren't free form, so this doesn't really matter. It also confirms everything, so it will never do something without you knowing. It also has the best voice I got to know. Natural, precise, short AND friendly; no clue why all these modern voices with way more compute all sound like garbage.