Faster to 1) leave the keyboard, 2) navigate with the mouse to the project pane, 3) select file, 4) back to the keyboard etc than to press 2-3 keys in Vim to move between NerdTree and your content and open whatever?
I don't think so.
In the very least, you could just use NerdTree with the mouse, exactly like TM, and it would be the exact same speed.
Some, particularly those who lean to the design side of things but still do a lot of coding, think spatially. For us, navigating the mouse to a specific point in space is faster, or at least less jarring mentally, than thinking of the name of a path to a file. The spatial and muscle memory takes care of switching to the file, allowing other high-level thoughts to stay in context.
It's the same issue I have with working in the terminal. Even after years I still find it very slow to navigate, because I always have to stop and think of the actual names and type them (partly) out, instead of letting my spatial and visual pattern matching do its thing.
Yes, there's been several UI research papers which show that keyboard heavy users underestimate the time their keyboard strokes take compare to quickly switching to mouse because of the higher cognitive load. I know, [citation needed], but I'm sure you'll find them if you google.