Salesforce CEO calls site outages unavoidable

SAN FRANCISCO--Salesforce.com Chief Executive Marc Benioff said Tuesday that a recent site outage was an embarrassment for his company, but that new technology will soon make its systems more resilient.

The Dec. 20, 2005, outage cut many companies off from critical data for hours on a busy, pre-holiday business day. It also called into question how well Salesforce, which stores customer and sales records for thousands of businesses, is holding up under rapid growth.

Benioff, who has commented little about the incident publicly, said in an interview at a media and customer event here that outages are an inevitable part of computing and that they happen very rarely at Salesforce.

"We don't want outages and we're doing everything we can not to have them, but we'll occasionally have them," he said. "That's part of computing...nothing runs at 100 percent availability."

Salesforce, based in San Francisco, claims an availability or "uptime" rate of between 99 percent and 100 percent. Yet a handful of customers that complained to CNET about the Dec. 20 glitch said smaller, less disruptive outages occur more frequently than they anticipated.

To combat such concerns, Salesforce has invested $50 million in new computing infrastructure designed to stay up and running in the event of a natural disaster. Benioff said during a speech Tuesday that the company will complete its conversion to its new systems, which include "immediate failover" capability, next month.

"It's a new level of performance, reliability and scalability for the company," he said during the speech. "Everything had to be rewritten. It was many, many steps we had to take and a huge amount of work over a year."

In addition to replacing nearly all its hardware and software and building two new data centers, Salesforce is adding database "mirroring" technology that will significantly boost its data redundancy capabilities, Benioff said. The mirroring system creates a duplicate database in a separate location and synchronizes the data instantaneously. In the event that one database is destroyed or disabled, the other one takes over.

But even mirroring would not have helped Salesforce avoid last month's snafu, which was caused by a database software bug, Benioff said.

"This will protect against huge natural disasters; we'll have zero downtime in a natural disaster," he said. "But it's not insurance against a bug."

[mottie]

salesforce?!

on cnets every 5th article is about salesforce. does cnet own them or something? who actually cares? my server had some downtime once too..

[Dachi]

Funny you say that.

Because I was thinking the same thing. I CNET searched for salesforce.com and counted 9 articles and blogs covering them in the last 17 days.

Few tech companies are active enough to be reported on every 2 days and I fail to see what is so appealing about salesforce that they would be an exception.

[BogusName]

They slam Oracle too

Who owns this site?

There is definitely some bias in the reporting.

[DavidWorkman]

Another bias ...

Yes, I agree. I see another trend too ... I would bet that their operating system is on unix or linux. I've noticed that whenever there's a story about server reliability, CNET (and others) won't mention the underlying platform if it's unix or linux. If it's a Microsoft platform, it always gets mentioned. Very biased.

[pdxtech]

They're Both in San Francisco

home town news i suppose.

[SWillington]

Sure... in a world of mediocrity

Marc, it's unfortunate that your company argues its limitations.

[scdecade]

Yep

It would wiser if he focussed on the actions Salesforce takes to ensure consistent, reliable service.

[pdxtech]

Just Jealous

sForce is displacing a lot of IT folks who were previously paid to keep their internal CRM systems going and probably love to say "I told you so" with every little 0.01% of downtime.

Setting peoples expectations that sForce is not perfect, but still up more than 99% of the time is a wise decision.

[chris sullivan]

It sucks

I run a sales team the sells to IT in the financial markets. It is way too slow but head mgt loves the report features (even though the report aren't accurate)