Overhauling Facebook's photo system
The Haystack system is custom-built but relies on content delivery networks and NetApp. Now Facebook is thinking less custom-built, more commodity components.
This was originally published at ZDNet's Between the Lines.
Facebook's photo storage system holds 850 million photos and costs a lot of dough. Niall Kennedy has a nice overview of what Facebook is doing to minimize its storage costs.
Facebook's system, dubbed Haystack, is custom-built but relies on content delivery networks and NetApp. Facebook is trying to minimize the custom stuff and use commodity hardware.
Kennedy does a nice job of synthesizing Facebook's storage system. In a nutshell:
• Facebook's previous system relied heavily on Akamai and Limelight to improve latency.
• That Akamai and Limelight use costs money.
• Facebook has invested in its own "blob" storage system designed to cut the total cost per photo on the social network's systems.
• The company hired a former NetApp engineer to redesign the storage system.
A lot of this storage architecture is complicated--and frankly over my head--but for the engineers in the house here's a presentation on Haystack from last year.
