Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm biased because I did some work on Colossus for a few years, but it's my opinion that Google's main advantage in its ability to build complex distributed systems is Colossus (and GFS). The Colossus layer is the point where a lot of systems actually take care of the CAP theorem, and because Colossus does it so well they almost don't have to worry about it at all.

Ceph, Hadoop, scale-out ZFS, and others that came afterward got the idea of distributing a filesystem but sort of missed the point that GFS/Colossus's great advantage was that it's so damn good at being a distributed system, and has been tried and tested through a lot of different kinds of failure modes.



Link to any resources you consider valuable for learning about Colossus?


Sadly, the best public material is the technical marketing for GCP, see the following for example (a few other materials can be found in this comments section):

https://cloud.google.com/blog/products/storage-data-transfer...

If you want to learn about modern exascale filesystems, the best paper today is probably from Facebook:

https://www.usenix.org/conference/fast21/presentation/pan




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: