I have a rather silly question - when designing UI features, how do UI designers test whether a design works as expected/is objectively good, and not just "Oh, this looks good to my eyes, others would probably feel the same way too!"? Off the top of my head, I can think of two ways - (i) have a private test audience, show various iterations to them, get feedback, and choose those features with highest scores given by the audience; (ii) get feedback from the end users, and make modifications as necessary. However, I'm guessing (i) is an expensive and time-consuming route, whereas (ii) could potentially lead to poor user experience and be detrimental for user retention. Is there a better way? In short, what's the "unit testing" equivalent in UI design?
That's not a silly question at all! There is an entire field of study around usability and human factors, better UI designers integrate at least some of this stuff into practice.
Sometimes it's required because you are in a regulated market: if you are designing a nuclear power plant or a radiotherapy machine or whatever, you are probably required to do some work to show your user interfaces are confusing in a way that can cause dangerous results.
Some of this stuff is expensive, you are right; on the other hand it follows a similar ramp up to other development work - namely that the later you discover something the more expensive it is to change.
You can also measure the end-user behavior without them knowing it. Metrics like "how long did it take users on average to fill in the form?", "how many received validation errors on this particular field?" can be used as proxies for how good the design is. Of course, this is only really an option for web applications.
