Ceph is a very interesting project. RADOS, their distributed block store is now mainline I believe and the project is coming along in leaps and bounds.

I'm unsure if anybody has a large scale RADOS based blob store though. It would be interesting to see how it holds up.