It either scales or it doesn't? You really underestimate the amount of effort needed to scale even the simplest website to millions of users. Even if your web app is written in OMG-uberfast x86 assembly, I can guarantee you that you'll have trouble scaling to Google-size traffic.
There are different levels of scaling. The more users you have, the harder it becomes. But it's generally not worth scaling beyond what you actually need because if you don't need it, then you've wasted tons and tons of man power that could have been invested in other areas.
Even so, there's a limit to how many nodes you can add given an architecture. Scale from 1 concurrent user to 1000? No problem, just add a server. Scale to 100000? Add many servers, and load balancers, maybe master-slave replication for database. Scale to 10 million? Putting more boxes in your data center help you anymore because your data center is full, you don't have enough bandwidth and even the master-slave database setup you had overloaded. You need to setup multiple geographically distinct data centers in the world and shard your database. What if you have 100 billion users? Adding more data centers on the same planet won't help anymore; I don't even know whether anybody is technically capable of serving 100 billion concurrent users.
Sure, the latter architecture can serve 1 concurrent user, but it's also 100000 times more expensive. The answer is not as simple as "it scales or it does not".
Otherwise a good read.