By that logic, you would have to open every single port on your firewall if you wanted to set up a web server. Fortunately, your OS just looks at the client IP/port to distinguish multiple TCP streams.