Yeah, that's true, and I did that when I built a 'streamer' from a Pi 2. It was great. At some point though the quality of the built-in jack did improve, and so when I built a sound/lighting effect box for a toy theatre with the Pi 4 it was plenty good enough to use, and made it simpler and cheaper to put together.

There are loads of cool projects for the Pi (synths etc) which chuck out audio and sound good enough through the built-in jack.