Is that really a problem? If something hits your sword it's going to make you rotate your wrist, not stop your hand, because it will be blocking the blade and not the handle.

Not if you're using a sword intended for cutting strokes and strike correctly. Distal to the "sweet spot", your hand will continue while the blade is rotated; proximal, and the distal end of the sword will tend to continue while your hand is levered backwards. Just as when using a tennis racket or a baseball bat, though, the moment is balanced around the point of contact when you strike correctly (the proximal part of the blade has more mass but less velocity; the distal has more velocity but less mass) and the blade, as a unit, stops (or is at least slowed by a uniform scale when the target can be cut through). Transitioning from a "chop" to a "slice" requires additional input from the swordsman.