Overhauling Facebook's photo system
The Haystack system is custom-built but relies on content delivery networks and NetApp. Now Facebook is thinking less custom-built, more commodity components.
This was originally published at ZDNet's Between the Lines.
Facebook's photo storage system holds 850 million photos and costs a lot of dough. Niall Kennedy has a nice overview of what Facebook is doing to minimize its storage costs.
Facebook's system, dubbed Haystack, is custom-built but relies on content delivery networks and NetApp. Facebook is trying to minimize the custom stuff and use commodity hardware.
Kennedy does a nice job of synthesizing Facebook's storage system. In a nutshell:
Facebook's previous system relied heavily on Akamai and Limelight to improve latency.
That Akamai and Limelight use costs money.
Facebook has invested in its own "blob" storage system designed to cut the total cost per photo on the social network's systems.
The company hired a former NetApp engineer to redesign the storage system.
A lot of this storage architecture is complicated--and frankly over my head--but for the engineers in the house here's a presentation on Haystack from last year.
