Telco error causes ISP outage

It only took moments for a Bell Atlantic Internet employee to accidentally cripple a fellow online provider's service. But it was 27 hours before the company's Net access was fully restored.

Bell Atlantic was processing a billing change Monday when a clerical error caused it to partially cut off service to its client, Mountain.net, an ISP based in West Virginia.

"The change was supposed to be made in billing orders only. But the appropriate flags were not put on the order and it was treated as a 'disconnect' order," Harry Mitchell, a Bell Atlantic spokesman, said today. "We're working to find out why it took so long to fix."

Mountain.net services nearly 7,000 customers, many of whom live in rural areas. Isolated from malls, movie theaters, and other public spaces, a lot of residents hang out online instead. Only about 500 Net users were affected by the lengthy outage, but the fumble shows that although people are becoming dependent on the Net, online access is sometimes available one moment and gone the next.

"It is incomprehensible that you'd have a service outage going this long," said Pat Clawson, who uses Mountain.net's email and Web hosting service for his software company, Telegrafix. "We depend on our Net site to sell our software."

From mistakenly slashing fiber-optic lines to overloaded email servers, rain storms, and software glitches, ISPs across the country have suffered from both uncontrollable service problems and outages that could have been avoided by the three Ps: proper, prior planning.

Last month, AT&T WorldNet system couldn't handle a surge in email usage, which led to unreliable service for a week, during which many of its 1 million customers sent email but did not receive messages. Yesterday, some Hotmail users couldn't log on to their free email accounts due to a "special system condition."

Almost always, the companies say only a "fraction" of their customers are affected by service glitches. But for those Net users, the time-outs can be very frustrating, making them prime targets for service guarantee campaigns from ISPs such as Sprint.

Offering reliable service is even harder for those ISPs that "peer" with major backbone providers to connect their customers to the Net.

"We know what it takes to reinstate a line that has been down. It's just matter of paging a engineer, and it usually takes place in about an hour," said Leroy Grey, president of Raven-Villages Internet, which sold its residential customer base to Mountain.net last year but still services a handful of businesses that were cut off by the Bell Atlantic mistake.

"This just makes us look bad, and we have no control of the situation," he added.

The owner of Mountain.net says he's glad he can buy Net access from a bigger carrier because he couldn't afford to put phone lines in the ground himself. However, when backbone providers try to cut costs, customer support services are the first to suffer, he noted.

"If there is an overall trend toward more errors with the phone companies, it's because they're are restructuring and regionalizing a lot of their technical support for Net access in order to control costs," said Mike Digman, Mountain.net's president. "They're losing close contact with their customers, so their technical support teams might not be familiar with our account or area. That's what I think happened with us."