The super-programmer that I'll hold up is Jeff Dean (see http://research.google.com/people/jeff/ for some of what he has done) and the task that I'll hold up is coming up with new standards and abstractions for creating scaleable software.
Sure, it is possible to break any one of the things he has done down into simple components. As a random example look at any of the many blog posts that discuss how MapReduce works and how to implement it yourself. (Heck, he had to do so when he turned his ideas into running software.) But could you have come up with those ways to think about problems? I am sure that I could not.
MapReduce is a simple realization that what we were doing in functional programming 20+ years before was useful in systems programming. Actually, "simple" isn't "easy" here, and MapReduce great realization so they deserve all the praise they get. But there was a lot of groundwork in place already.
A great number of advances in math and computer science can be recognized as someone taking techniques developed in one area and recognizing how they can apply to different area of programming. MapReduce is clearly one of these, and they were quite clear where they pulled inspiration from.
However Jeff Dean was the source of a number of insights which were utterly essential to Google's managing to scale. Only some of these have been widely disseminated outside of Google.
Once lightning has struck enough times in one place, you can rule out coincidence. It is clear that intellectual lightning keeps striking at a place named Jeff Dean. And the fact that it does is part of what makes him such a great programmer.
I would give Sanjay some credit also...and probably many others. Google has a great formula, but let's not get carried away; there are a lot of smart guys in systems who are very good at what they do. But super programmers is just a bit too condescending, I don't think any of them would appreciate that label.
Jeff Dean is famously modest, and undoubtably would resist the label "super programmer" in the same way that Terry Tao resists the label "math prodigy".
That doesn't mean that the label does not fit.
And no, he doesn't work in a vacuum. On the other hand, I think that he deserves the praise that I just heaped on him.
A great number of advances in math and computer science can be recognized as someone taking techniques developed in one area and recognizing how they can apply to different area of programming.
You probably already know this, but your insight isn't unique to programming. Virtually all fields progress this way, and Steven Berlin Johnson wrote Where Good Ideas Come From, which is substantially about how, where, and why this kind of borrowing takes place.
It's like math. You don't believe the difference between the best and the rest until you meet the best. It is apropos that someone linked to a Terry Tao math post in this discussion, because right now, in math, Terry Tao is definitely in the "best" category. Unless we have chased the people like him away from programming, they must certainly exist in our field as well. However, our works are largely anonymous, which means that it is much harder to tell who is the best.
Indeed; I thought I was very good at math until I landed in a course where there was this guy who just instantly grasped everything and it's consequences without having ever seen the subject we were taught. It was great talking/working with him but it deterred me from pursuing math as major; I reasoned that if there is someone so incredibly brilliant that he can replace this entire auditorium by himself, what am I doing here? I was very young at the time and I now know that brilliance by itself is not enough. He is not working in math now but is a programmer somewhere; so him replacing the entire auditorium for math didn't happen.
For super programmers (definitely there are plenty of them like Carmack) this of course does not fly; one super programmer can maybe do the work of the entire auditorium, however the rest of the auditorium is needed as well as there is a shortage. Which is different from math, physics etc.
> But I do have a challenge for you. Surely you know a “super programmer.” You might even be one. If so, let’s hear from you or them. Tell us that complex problem that is too difficult for the rest of us to possibly understand.
"There are only two hard problems in Computer Science: cache invalidation and naming things." -Phil Karlton
I'm a little hesitant but inclined to agree.
I hesitate because like most discussions on these matters, the terms are defined differently for everyone I talk to and that makes agreeing a little difficult without qualifying a lot of statements.
I'm not sure I've ever come across something that was simply too difficult to understand, per se, just things that take different amounts of time and different amounts of diligence to understand or implement.[1]
A lot of content in programming (and math) is what I like to call "NP-understanding complete". Problems that are difficult to find solutions for but once shown a solution are very easy to comprehend. Coming up with novel solutions to these problems in the first place is admirable, and I would be inclined to call those who do "super", but I don't know if there are very many "super" people who consistently do this. It may well be that finding novel solutions is merely a product of spending a lot of time at the frontier of something, and in fact its not only not a gift, but difficult to see them coming (the solutions) until you have them!
[1] When looking at superhuman accomplishments or worrying that I might not understand some subject (something I was completely frightened of as a child) I'm always reminded of The Ladder of Saint Augustine by Longfellow.
The mighty pyramids of stone
That wedge-like cleave the desert airs,
When nearer seen, and better known,
Are but gigantic flights of stairs.
The distant mountains, that uprear
Their solid bastions to the skies,
Are crossed by pathways, that appear
As we to higher levels rise.
Sure you might be able to understand something given enough time and guidance. But to me a super programmer is someone who is able to make that leap and create something novel, useful, and valuable. For example, John Carmack, who many would describe as a super programmer, showed the rest of the game industry what could be done with 2.5D in Doom, and later commodity 3D hardware with Quake. You can certainly review his code since it's open source and understand it, but how many people could have independently done what he did when he did it? Not many I would wager.
The answer is one: the guy who spent most of his time honing his skill, and mastering his domain.
It's that simple.
No one can solve a problem unless he/she knows the problem inside out and has the pieces to solve the problem (a.k.a previous experience: success or failure).
You can't put someone who is not familiar with the problem domain and expect him to do superb job given a deadline. If he can solve it under-pressure and on-time, chances that he has previous experiences or building blocks that can help him greatly.
The problem with most people is that they're focusing on the wrong thing because of many reasons: distraction from family, distraction from the things they read, tend to spend too much time viewing the problem from the same perspective instead of taking some time off to think about it from a different angle.
This is nonsense. His key argument is that hard tasks are reducible to simple tasks, and therefore they are actually not all that hard.
Ask anyone who's implemented, say, Paxos, and ask them whether because the individual steps were easy the whole system is therefore easy. Or indeed any other distributed system. Just because something can eventually be decomposed into simpler things does not mean that it is a fundamentally simple construct, nor does it mean that a programmer who can manage the simpler constituents is competent to handle the overall system.
The author blows his own argument out of the water when he says:
> I would venture to say that really good programmers are about 10 to 20 times as effective as just average ones.
Aren't those the super ones? Or at least on that track?
But disregarding that, I'd personally say the super programmers are the ones capable of inventing git, or bitcoin. I don't care what you say, 99.9% of programmers simply couldn't do that. Saying super programmers don't exist is like saying Einstein's theory of general relativity is decomposable into easy parts.
Most ideas can be broken down into simpler parts. For instance, multiplication used to be a university level subject, but most literate people can do multiplication up to any digit (given the time, of course).
Just because you understand something deeply and possess the pedagogical skills to explain it to an elderly layperson doesn't mean it's suddenly easy after you've explained it, and doesn't mean it was easy to put in the hard work required get to that level of understanding. Breaking something complicated down and explaining it to a layperson is a different thing than actually doing the work yourself.
Are you suggesting that if one is able to explain a complicated problem to their grandmother, somehow the original problem was actually easy, or is made easy by this explanation? I haven't read Einstein's book on relativity linked in your sibling comment, but I don't think the existence of this book makes relativity an easy thing to have worked on.
I agree, someone who is 10-20 times more effective is a big deal. Imagine if we had pilots who could fly you to your destination 10-20 times faster; farmers who could produce 10-20 times more food; or civil engineers who could build skyscrapers 10-20 times taller.
There aren't many fields where a 10-20x improvement is feasible.
"Superstars" seem different to me not because they can or can't decompose problems into steps, but because they seem to have a 6th sense for what is possible.
Everything is easier once someone has done it before. You don't even have to know how it was done, just knowing its possible is enough. Superstars can substitute some sort of magic pixie dust(1) for this foreknowledge and proceed like they already know its been done and they're just rediscovering the minutia of making it happen.
Its humbling to me when I meet them because they think thoughts and have insights that I know I would not have had even if I worked on the same problem for 100 years.
(1) I strongly suspect that its made almost, but not quite, entirely of crazy.
Sometimes I am in superstar mode and more often I am not. When I am in superstar mode, I am able to latently process problems ahead of time and decompose them. That way when I get to the problems they all seem simple.
The rest of the time I have to run up against a problem first, let it sit and see if it decomposes for me, and if not come back and actively try to decompose it. Once I am actually ready to sit down and start coding, all the hard part is over. The question is how much of the hard part can be done latently before I get to it.
I think this kinda goes along with the four stages of competence, when you are at a level of unconscious incompetence in any subject matter, it all seems like black magic to you, you've no idea how it works and it seems like the most difficult thing in the world, but just a little insight into how something works can open your mind into understanding it enough to see that like most things, it just takes time and effort to learn just about anything.
Taking complex things and making them simple is not the mark of a "super programmer". Or, at least, it's not enough. A highly skilled programmer needs to be able to deeply understand multiple layers of abstraction at the same time. He or she needs to be able to know what an abstraction layer promises, and what its performance characteristics are --- and also be able to grok the layers above and below that abstraction layer to a similar deep level.
I call it "abstraction surfing", and it's not something that all people can do well. I believe having this skill tends to correlate very well with high productivity programmers --- and it's not a skill that all people have.
its either you are a superstar problem solver or a simple problem solver. the programming is just a means to an end. a superstar problem solver has the ability to take most problems and break them down and find the means to solve any problem, weather related to programming or math or most anything else.
superstar problem solvers can jump into most fields of software and learn how to cope in their new environments. every job i've had has forced me to learn a different language and work on a completely different problem. the problems may change, but how one attacks and breaks them down them rarely does.
> But I do have a challenge for you. Surely you know a “super programmer.” You might even be one. If so, let’s hear from you or them. Tell us that complex problem that is too difficult for the rest of us to possibly understand.
I generally tend to agree, and in the majority of computation I think this is generally true. Compilers for instance I think are one thing that the majority of programmers imagine to be simply above them. While in reality they are much like the rest of the software we write.
However as a counter example, the hardest things I've ever written included a rather complicated lock-free data structure, specifically optimized for 3D effects rendering and simulation. The amount of mental capacity that took was pretty astounding, and what I took from the experience is that I'd try to never have to do something of such complexity again. I am also, however unsure if I could complete a similar task again. It did give me an appreciation that some things are just simply harder.
I think this post has good intentions. There's some truth to it and there's definitely value in plumping yourself up from time to time. No one should be worrying about whether they're a "Real Programmer" or not. I agree that there's no value in beating yourself up or beating up your work.
That said, I think the post takes it too far, and veers off into self-congratulatory back-patting that I think is not valuable, at all.
I also think the author is wrong about hard problems, or at least managed to come up with a tautology that's not very interesting. Sure, many (and perhaps all) hard problems can be decomposed into smaller, simpler problems. But that's often not trivial, and the decomposition step often requires a lot of hard work. Even if hard problems can be broken down into smaller pieces, that doesn't always make the problems easier, even if it makes them more manageable. They still might require tremendous effort.
The example given isn't a very good one; that sum just isn't that advanced mathematics. Now that's not to say it's not Greek to some programmers, and not to pooh-pooh people who have a hard time with math; I have struggled with it a lot. But there's a lot of math that really is hard to grasp, and isn't just a question of learning the notation, which is what the sum example is really about.
The challenge sets up a straw man, IMHO. "Tell us that complex problem that is too difficult for the rest of us to possibly understand." It's asking the reader to be an arrogant ass.
"I’ve just never seen a problem that couldn’t be broken down into simple to understand components, yet."
Again, I think this just isn't a useful criteria. Broken down into simple to understand components doesn't make things easy. Here's an example, from my own dabblings: I am absolutely fascinated by programming language theory and compilers. Recently, I've been trying to get at the heart of two sub topics: parsers and types. I've been struggling to find presentations of parsers and parsing that really peels back the magic and shows how everything works, without just punting and letting Lex and Yacc do the dirty work (btw if you know of a compiler text that doesn't do this, let me know!) Yes, you can break parsing into a lexing step built up from regular expressions. Yes, you can translate regular expressions into NFAs and convert those into DFAs... and so on. But putting this together into a lexer generator is non-trivial. It's not easy, even if the components are simple. It's the same with type theory. You can write simple syntax directed type checkers and they might fit in your hand, but if you really want to understand type theory, it turns out there's a lot of math, and proofs to get through. Sure, you can decompose that into topics from set theory and so on, but this doesn't make it easier.
More and more, I think his core argument is just really flawed. There's some connection here with Rich Hickey's "Simple made easy" talk, where he argues that simplicity can be hard to come by, and how easy and simple are different... but I've written too much and will leave that as an exercise :)
This is a good point. Actually, "simple" requires understanding and insight that is not "easy" at all. Sometimes you have to go with a complex solution simply because you do not understand the problem well enough, but the understanding you get doing the hard thing can lead to enlightenment that leads to beneficial decomposition and simplicity.
All solutions to the hard math problems only look easy in hindsight, it probably took frustrating long while for the super programmers to get there. This is why Google hires a bunch of PhDs to dev, they are just conditioned on slogging through those problems given their research training; they are not necessarily better than the non-PhD programmers, they just often spend their time differently. Great at math, then you can focus on math problems. Not great at math, you are probably great at something else that needs to be done. Diversity is good.
As an aside, real compiler writers :) write their own lexers and parsers. Once you've written a lot of these, you get more efficient at writing more, but its a 10,000 hour thing.
Yes, for example. Take the area under a curve of a known equation.
What we do is divide it into an infinite number of rectangles we can add together and come up with a function for that. Rediculously easy in hind-sight but that's the basis of the integral. Of course using computers those which cannot be easily processed symbolically can be processed using Euler's Method, which gives you a close approximation.
Edit: Also I think that all the complexity of accounting systems came from small solutions to questions, such as tracking how different business partners are to be compensated in a Europe where nobody had the money stores to buy much without credit. See http://ledgersmbdev.blogspot.com/2012/08/a-software-architec... and I think that has a lot of lessons for programming today.
I've been struggling to find presentations of parsers and parsing that really peels back the magic and shows how everything works, without just punting and letting Lex and Yacc do the dirty work (btw if you know of a compiler text that doesn't do this, let me know!)
That explains how to write a top-down (or recursive-descent) parser. Humans generally find it easier to understand and write top-down parsers.
Tools like YACC exist to make it easier for humans to specify bottom-up (or shift-reduce) parsers. They have a number of benefits over top-down parsers including performance.
If you want to understand the magic of YACC, without the whole "it's magic" bit, you need to understand how to write shift-reduce parsers.
Recursive descent is nice if you want to finesse your error recovery, which is much more important these days than raw parsing performance. Times are a changing...
Ahh, I have not really dug into shift-reduce parsers, but I am finding the wikipedia article to be surprisingly well written (once you get to the portion about the gramar for the example..)
I have Scott's book, and I've read his coverage of parsing, although it's been over a year since, and TBH I don't think it's very good. The whole book just tries to cover too much, and ends up doing a fair amount of hand waving.
It covers a lot, but I didn't see much handwaving about the basics of parsing. Particularly the sections that apply to the functionality that people would use lex and yacc for.
A lot of people have commented that they do not agree because most people could not come up with Git, invent (from other ideas) modern-day MapReduce, etc.
I don't agree that the article is wrong for those reasons. Those people who invent those things are able to do so because they have spent years learning the underlying principles. And this does not only apply to programming; it applies to any scientific field. Somebody mentioned Terry Tao; well I would say that Terry Tao is a "superstar" because he has combined talent and more importantly because he has studied maths for years.
Talent + (a lot of) practice = superstar!
Look at the story of Andrew Wiles's proof of Fermats last theorem. He is undoubtedly a talented person, but was able to complete that work because he had studied math for a long time and combined the insights of many fields into his proof. Practice and talent makes a superstar!
This discussion enters into the realm of "myth" so much more often than fact, because in these discussion, no one even tries to define what "average" is. Is a student studying Java in high school for the first time a programmer? By average, do you really mean the level of programming skill that you meet with the highest frequency or the actual mathematical average? How do you even measure "productivity" or "effectiveness?" I bet one can define parameters where the best programmers are only 3x as "productive" as the average programmer.
And I'm not saying such things can't be measured. I'm just saying that discussions about these myths are often devoid of such measurements.
In my view, discussions like this are much more about the human tendency to deify than an objective search for truth.
Actually that code snippet that computes a summation can be done without any loop or recursion at all:
function sum (m, n) {
return m * (m + 1) / 2 - n * (n - 1) / 2
}
I only know this because I remember from math competitions in high school learning that you can compute the sum of the digits from 1 to n quickly from `n * (n + 1) / 2` which naturally follows from the equations regarding binomial sums.
This example somewhat agrees with the author's main point that we are more a product of our experiences than endowed with certain innate capabilities.
The problem that I have with "programmer" is that there are a lot of clueless people that call themselves "programmer", and I think there is a clear boundary. Given the population and size of clueless programmers, I'd define the non-clueless programmers as super any day.
The super programmer does not necessarily solve a different category of problems. She sees maintanable and performant abstractions when she encounters new problems. She sees order where others see chaos.
1. There is a law of diminishing returns when it comes to effort spent vs reward. I think we can take that as axiomatic;
2. Different skills are naturally harder or easier for different people. Call this affinity;
3. Given the effort vs reward curve and the different affinity levels different people have you will find that with any given skill people will tend to plateau at a particular level.
This isn't simply a question of intelligence or innate ability. More than anything it comes down to motivation. Priorities vary. Perhaps someone is particularly keen on their social lives, families, hobbies or whatever and whatever skill from which they derive their livelihood just needs to be sufficiently good for them to get by.
For example, I was an OK chess player. Not great. Not even particularly good but I did reach a point where I recognized that if I wanted to get better it was going to be a serious amount of effort. Nowadays I tend to play Euro boardgames simply because they have a nice effort vs reward curve and aren't (potentially) demanding mistresses like, say, Chess, Go, etc;
4. The more time you spend in any particular field the more you will master it and begin to see problems with it; and
5. Certain environments will tend to make you more or less likely to succeed. For example, if you want to get better at pretty much anything, hang around with really good people who are really good at it.
Many people don't get that far because they either plateau or the effort vs reward curve gets too flat and they move onto other things.
So what I tend to think of as "super programmers" combine the traits of:
- some level of natural affinity;
- the dedication to stick with something; and
- they care enough not to simply to just enough to get by.
The last I think is particularly key. When I assess an engineer one of the things I'm looking for more than anything is that they care about their craft. They can assess what they see and what they do critically and recognize where it falls short, what works and what can be done better.
It's part of the reason I'm a strict pragmatist and prefer other engineers who are like-minded. Ideologues (IMHO) tend to have judgement clouded by dogma and preconceived notions.
Woody Allen famously said "80% of success is just showing up". If you care about your craft and keep plugging away at it good things will happen. Those programmers we tend to think of as "super" or "rock stars" I think tend to combine all these traits with the timing and luck of having that flash of insight at the right time and following through with it.
Like they say, good ideas are cheap. It's execution that matters.
Very thorough explanation that I definitely agree.
I love watching Soccer and Basketball. In those two fields, the best players are those who dedicate themselves to the game.
Lionel Messi, Christiano Ronaldo, Michael Jordan, LeBron, Kobe. The list goes on. They have the same trait: passion and dedication. Some may have born with the skill (Ronaldo, LeBron). Some takes time to develop (Jordan, Messi).
Having said that, our field is a little bit strange because there's a divide between being good at a particular set of skills and being jacks of all trades.
For example: we tend to argue and discuss the topic of how learning new programming language can change the way you think about a particular problem and hence make a conclusion that those who put the effort and time to learn niche language such as Erlang, LISP dialect, Ruby, Lua get this automatic recognition as a "better programmer". (They may be a "better" programmer if we compare it to John Doe Programmer, but are they truly a better programmer?)
On the other side of the coin, we have people like James Gosling, Bjarne Stroustrup, John Carmack, Dennis Ritchie, Joshua Bloch, Guido von Rossum, Kent Beck, Martin Fowler, etc. that are pretty much sticking with one or two languages but truly focus on their problem domain (as oppose to the languages that they use).
There seem to be a group of people who are good at plumbing different technology and get something out of it and there are another set of group of people who specialized in specific technology but they're just... awesome at it.
Example: A typical blog article explaining the full-stack of technology => "We use Rails as front-end, MongoDB as our back-end, memcached for our caching layer, ActiveMQ for our queueing system" vs "Here's the architecture of Mailinator: Java and Tomcat" (http://paultyma.blogspot.ca/2006/12/architecture-of-mailinat...)
PS: Kent Beck and Martin Fowler may not have the "proper" HN street cred but they are definitely put their sweat, effort, and time to get better at what they do (OOP, software design/development, software process).
Out of curiosity why would Kent Beck and Martin Fowler have "proper" HN street cred? I would have thought it safe to say that they had both have had revolutionary insight into their areas of focus.
One quality I find with above-average programmers is they type so much faster than the average programmers.
100-140 WPM on average.
What's the importance? They code quicker, hit walls quicker, debug quicker, fix quicker, ship quicker. They also have to be able to think quicker once the steps ahead of them are clear.
Getting through this cycle quicker I think makes a programmer much more effective at building lots more software, and through it, finding the lessons from the experiences a lot quicker.
Input speed is totally a major part of being a productive programmer. Not necessarily raw words-per-minute typing (it helps, but how much of programming is words, compared to symbols?) But navigation speed around your machine and environment makes a huge difference. Things like cutting a block of code and jumping down two screens to paste it. Or memorizing all the keyboard shortcuts for your debugger, so your mind stays on the problem rather than nosing around toolbar buttons. Or operations as mundane as fixing the indenting of a block of code, that a programmer could easily do a hundred times in a day. Save five seconds on that and it seriously adds up.
I naturally zip around (Windows) environments very fast. I can typically compile and alt-tab over to the output directory and copy and paste to a deployment folder in under two seconds, often before the programmer watching next to me registers that I've done it. This seriously matters! Like you say, programmers with this practiced skill set can code quicker and debug quicker and ship quicker. (There are downsides; I can't stand pairing with a slower typist, drives me crazy.) Laptops are right out for serious productivity for me, for this reason. My hands know all the (Windows) keyboard shortcuts in the Ins-Del-Home-End block of keys, so my navigation speed on a laptop keyboard without that usual layout drops to half or worse.
Actions per minute as a metric can go beyond Starcraft. It matters for programming too.
I dearly wish to see the day when neural human-computer interfaces become readily available and accurate. Imagine just thinking 'click' at a toolbar button and there it goes. Not just for typing and programming either, but how about games? Imagine playing Starcraft at the speed of pure thought, with no APM constraints to get in the way...
> One quality I find with above-average programmers is they type so much faster than the average programmers.
Correlation, causation, etc... I imagine that most above-average programmers have spent a lot of time programming, which does involve some typing. So they get relatively fast at it. But I would just caution against bringing up typing speed since it shouldn't be a goal in and of itself.
> They also have to be able to think quicker once the steps ahead of them are clear.
Not really sure how typing quickly helps you think quicker. I've seen programmers who appear to love banging their fingers against the keyboard but never stop to think, and suffer for it. (anecdata, etc)
The connection I was making is Above average programmers seem to do everything quicker, including typing. Didn't mean to imply it was 100% of the time, or the other way around that people who type fast are probably good programmers -- it's just one observation point that I compare their output with.
Another post here mentioned the idea of 'actions per minute', I like that idea a lot.
If you just do a whole lot of typing you will tend to get faster, so if this is true, I'm not sure it wouldn't just be because of experience writing code.
He's wrong.
The super-programmer that I'll hold up is Jeff Dean (see http://research.google.com/people/jeff/ for some of what he has done) and the task that I'll hold up is coming up with new standards and abstractions for creating scaleable software.
Sure, it is possible to break any one of the things he has done down into simple components. As a random example look at any of the many blog posts that discuss how MapReduce works and how to implement it yourself. (Heck, he had to do so when he turned his ideas into running software.) But could you have come up with those ways to think about problems? I am sure that I could not.