Most lexers just match the longest legal token, so there is no ambiguity between = and == and the grammar does not influence tokenization.

The issue with JavaScript is that say /a+”/ tokenize differently based on the position in the grammar, so you can’t tokenize in a seperate pass before parsing.