Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Designing a Scalable Twitter (natishalom.typepad.com)
14 points by twampss on April 15, 2009 | hide | past | favorite | 6 comments


I don't understand why he puts numbers on the scaling requirements. Either it scales or it doesn't.

Otherwise a good read.


It either scales or it doesn't? You really underestimate the amount of effort needed to scale even the simplest website to millions of users. Even if your web app is written in OMG-uberfast x86 assembly, I can guarantee you that you'll have trouble scaling to Google-size traffic.

There are different levels of scaling. The more users you have, the harder it becomes. But it's generally not worth scaling beyond what you actually need because if you don't need it, then you've wasted tons and tons of man power that could have been invested in other areas.


Scalable, by definition, means more nodes = more capacity.

Speed has nothing to do with it.

A simple stateless website that requires no shared data like from a db, is scalable, since you can add more web servers to fulfill more requests.

Using memcached, however, isn't scaling, because while memcached itself does scale, if the backend it reads from doesn't, the whole system doesn't.


Even so, there's a limit to how many nodes you can add given an architecture. Scale from 1 concurrent user to 1000? No problem, just add a server. Scale to 100000? Add many servers, and load balancers, maybe master-slave replication for database. Scale to 10 million? Putting more boxes in your data center help you anymore because your data center is full, you don't have enough bandwidth and even the master-slave database setup you had overloaded. You need to setup multiple geographically distinct data centers in the world and shard your database. What if you have 100 billion users? Adding more data centers on the same planet won't help anymore; I don't even know whether anybody is technically capable of serving 100 billion concurrent users.

Sure, the latter architecture can serve 1 concurrent user, but it's also 100000 times more expensive. The answer is not as simple as "it scales or it does not".


Fair enough, I just wish articles would stop using the word 'scalable' to describe everything.


I think Twitter is a bad usage case, because it's not something that NEEDS to be scalable.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: