A week after a slew of popular Web sites were knocked offline temporarily by a power outage in San Francisco, the data center for the Web sites has finally disclosed what went wrong.
Basically, backup generators made by a Dutch company called Hitec failed to work, according to an update posted to the Web site of data center 365 Main.
Here's what happened: transformer breakers at a San Francisco Pacific Gas & Electric power station unexpectedly opened, causing a power surge that cut off electricity to a portion of the city. (PG&E does not know why the breakers opened.) Three out of 10 of 365 Main's 2.1-megawatt backup generators failed to start up, cutting power to equipment of more than 40 percent of the data center's customers for up to 45 minutes.
And why exactly did the backup generators fail to, well, provide backup like they were supposed to? According to the technical mumbo jumbo, the problem was attributed to "weakness in a small but essential component in the back-up system known as a DDEC (Detroit Diesel Electronic Controller)...(there was) a setting in the DDEC that was not allowing the component to correctly reset its memory. Erroneous data left in the DDEC's memory subsequently caused misfiring or engine start failures on the next diesel engine call to start."
Not to worry, Craigslist, Yelp and the others, you'll get refunds on your rent for the inconvenience.
Even with the outage, the San Francisco facility offers 99.9942 percent uptime, the company said. Personally, I'm holding out for five 9s.