Or more accurately, because you didn't say otherwise in the System Preferences.

I dont see in Big Sur any system-wide option to close an app after closing the last window.

There is only the opposite: "Set all app windows to close when quitting apps"

I currently use a 3rd party app to close for me apps when I close all of their windows.

I dont see why this option should exist in the first place. Let alone being the system default.

edit: Is there an option now in the MacOS Big Sur to enable this? I have a 3rd party app to achieve so.

To save loading/startup time. Also has to do with oldschool Macintosh usability, which is a important feature that veteran Mac users want to have.