I had a co-worker who coded his own CMS for a relatively successful web agency (it was the pre-wordpress era). And he coded part of this CMS on paper at a party. Must have been one crazy party :)

I think most people who do testing themselves are too lazy to do this or even grasp the importance of AB testing tool and Analytics tool integration and post-test analysis. The few that do understand will thrive though :) Thanks Peep for the incredible insight into how the pros do it