I think the problem is you expected to do stuff with your finger. When I need to go somewhere, I just tap the mic button (or vlingo on my old droid phone) and say "Navigate to best buy" or whatever. Then the only finger touching I need to do after that is if I want an alternate route due to traffic or something, and that's an obvious button right there on the screen.
Really, with the voice recognition stuff available for android these days, I barely even type any more. It's all voice control and text to speech.