Ultimately that leads to Popper's Paradox of Tolerance.[1] Do you defend the absolute freedoms of those whose goal is to destroy that freedom, along with you and many others with it? If yes, how do you stop them from accomplishing their goals? If no, where do you draw the line? (To be clear, I consider these critical but ultimately rhetorical questions with no obvious good answers.)
1. https://en.wikipedia.org/wiki/Paradox_of_tolerance