Vandals blamed for phone and Internet outage

Update 2:58 p.m. PDT: This story has been updated with information about what caused the massive phone and Internet outage in Silicon Valley on Thursday. Comments from Sprint Nextel have also been added.

Vandals are to blame for the massive phone and Internet outage in Silicon Valley on Thursday, an AT&T representative has confirmed.

A story published by the San Francisco Chronicle and carried on SFGate.com first reported that police confirmed the phone and Internet outage that has left thousands of customers in the San Jose, Calif., area without phone or broadband Internet service was caused by vandals who had cut fiber-optic cables.

Police told the newspaper that four AT&T fiber-optic cables were severed shortly before 1:30 a.m. PDT along Monterey Highway north of Blossom Hill Road in South San Jose. A cable in San Carlos, Calif., owned by Sprint Nextel was also cut about two hours later, Crystal Davis, a Sprint spokeswoman confirmed.

Davis said that a manhole cover had been lifted, and the fiber underground had been cut. She confirmed that the Sprint fiber that was cut also appeared to be the work of vandals. But she explained that fiber cuts happen all the time, typically due to an accident.

"Fiber cuts happen more often than people think," she said. "Usually it happens accidentally when someone is drilling in the ground, landscaping a lawn or repairing some other infrastructure in the ground. We know this happens all the time, so we're ready to reroute traffic whenever we have to."

Earlier AT&T confirmed the incident and said it's working to resolve the problem. The company added that in addition to repairing the cut cables, it's also looking for alternative options to reroute traffic and get customers back online. And the company expects service to be restored in the new few hours.

"We are aware of a cable cut situation impacting services in Santa Clara and San Jose areas," a company spokesman said an e-mail. "We have crews on the scene. More details and repair ETAs will be shared as soon as possible. We apologize for any inconvenience this morning's service outage has caused."

The company spokesman also said AT&T is working with law enforcement officials to find who vandalized the cables. The company declined to comment further on how vandals were able to gain access to its cable infrastructure.

The fiber outage is affecting service for customers of AT&T and Verizon Wireless and Verizon broadband customers. Verizon uses AT&T's fiber-optic lines to connect its wireless and DSL service to the phone company's own national network. So a disruption in AT&T's fiber link also affects Verizon's service.

A source close to Verizon said the phone company has its own technicians on site to help AT&T repair the cut fiber as quickly as possible.

Sprint's Davis said that the Sprint fiber that had been cut was used to carry traffic for Sprint's IP data customers, most of whom are business users. Sprint was able to reroute the traffic and have service restored to most of its customers before business hours this morning.

But like Verizon Wireless, Sprint also uses AT&T's network to carry its so-called "backhaul" wireless traffic from its cell towers to its own national network. As a result, the AT&T fiber cut has also interrupted service for thousands of Sprint wireless customers.

Davis said the company is working closely with AT&T to get service restored as soon as possible. But the company is also considering deploying mobile cell towers or cell-on-wheels units that will provide temporary service to Sprint's wireless customers.

"We have resources that can provide supplemental service staged around the area that has been affected by the outage," she said. "But we are waiting to hear from the local exchange carrier (AT&T) about when they think service will be restored."

Davis said Sprint will likely decide whether to use mobile cell stations in the next three to four hours.

AT&T hasn't provided any additional information about when it expects to get service working again. The last update the company provided over its Twitter feed, ATTNews was posted more than three hours ago and said: "Our priority is to restore service to all affected customers as soon as possible. We apologize for any inconvenience."

At least one company affected by the outage has said it's back in business. VerticalResponse, a San Francisco-based company that uses an ISP whose servers are collocated in Palo Alto, Calif., said its servers went back online around 12:30 p.m. PDT. The company, which works with about 56,000 small business customers to distribute direct e-mail marketing campaigns, has been keeping customers up-to-date using its Twitter feed VerticalResponse.

"Our systems have been restored and delayed campaigns are in the process of being sent. We sincerely apologize for this inconvenience," the company's latest update reads.

A report earlier Thursday from KTVU-TV in Northern California said the service outage is affecting customers in southern Santa Clara, Santa Cruz, and San Benito counties. A dispatcher in Santa Clara County told the TV station that the line was knocked out at 2 a.m. PDT.

[n3td3v]

Conficker Working Group website is one of those affected by the fiber cut.

[totocalimero]

Hmm... Tha'ts very strange. Could this be a "real life" test to assess the effectiveness of cutting off a backbone in order to shut down the internet in case the President assumes dictatorship powers?

[qnet]

Forget your meds today?

[sythara]

ahahahaha

wow. Seriously?

Did Alex Jones come up with that one for you?

[JCPayne]

I personally believe it could have been caused by AT&T themselves. Just by their actions they appear like their over reacting. Why a reward? How is this diff. from othere fibre cuts where customers suffered from downtime? If I recall correctly according to a special that was on the History or Discovery Channels, AT&T had taken various decisions to take their fiber cables that are in California and splice them and redirect it to monitoring (spy) equipment. I believe this may have been AT&T/NSA's chance to upgrade or disconnect that.

[jillianb_00]

This affecting a business in North Texas also. Out parent company is in San Jose. We have no email and no access to servers at the San Jose site.

[ThomasKeeley]

I noticed that our email system, Vertical Response was also out. You'd figure that they'd at least have some sort of redundant connection to at least enable them basic functionality should something like this happen.

I can handle being without email for awhile, but what really bothers me is that 911 services were down.

I just read an article yesterday talking about the issues with our power grid, and I cant help but wonder what security upgrades are needed for communications systems. I would hope that those at Homeland Security and Health and Human Services are looking into this to see how they can prevent it down the road.

[sythara]

There should be a reduntant system. After all thats how the internet functions, if connection to one router is lost, another one picks up the workload. Unless I'm misreading the article.

[cerving]

It's the Internet backbone that's redundant, not the "last mile". Unfortunately, the "last mile" goes a lot further than a mile, and affects a lot of telco central offices. It's basically a star topology from the Internet backbone routers to the end users, so an outage at a major CO can knock out a wide area.

[vanheve22]

"But if they are unable to get service, they should get to their closest fire or police station or local hospital as quickly as they can."

Such valuable info

[Hamlet]

There is a simple explanation: union workers, who are no longer working under a contract, cut the line. That is how they were able to "access" the cable infrastructure, they alreayd had access. The CWA contract expired midnight this past Sunday morning.

[Ezravision]

I hope that is not what happened. I don't want anything to sour the contract negotiations, and there are plenty of AT&T union workers who want to be civilized throughout the process. I take my job seriously as one who protects copper and fiber optic facilities in the east bay during digging projects. If your explanation were true, that would be bad news for us all. Most vandalism damage I have heard of is done to pole-attached copper for recycling value, so this does seem unusual.

[chukra]

That would be the cover story. Probably, actually an exercise to discern impact if and when Obama does shut down internet under Jay Rockefeller's newly introduced Senate Cybersecurity Act of 2009.

This would create yet another government bureaucracy, the Office of the National Cybersecurity Advisor, effectively giving Obama a "Cyber Czar" in the Commerce Department and the power to shut down the internet.

[losingmyjob]

That's nice. Just what we need now is to have people blame it on the union workers. If it is a member of CWA, they are acting on their own and do not represent any of the 112,500 CWA members who are now working without a contract. They would be causing great harm to all of us, as well as the company. I understand that some idiot who is an union employee could have done it, and deserves to be arrested and thrown in prison for a long, long time This is a terrible crime. But It could be someone who wants to sabotage the union, knowing that the CWA would be blamed first. Or it could have nothing to do with the contract. There isn't a simple explanation, and it's being investigated as an act of terrorism. Which it is no matter who did it. 911 was put out of service. The way you phrased your comment is degrading to 112,500 people who are scared about our futures. We are loyal to the company, some of us have been there over 40 years. We are very skilled and knowlegable. We are going to work and doing our jobs just as before. We are real people, not just "union workers" who are one entity conspiraring to take AT&T down. And here's a big surprise for you - our managers are not "the big bad corporation" to us either. They are our co-workers and even friends, and they were glad we came to work this week. We care about each other. Before you speak give a little more thought to your words because you have personally attacked a huge amount of people.

[emcourtney]

On the other hand, direct action gets the goods. They keep this up and the bosses will be screaming for a contract in short order.

[AndrewRich]

Odd, I have Sprint wireless (and live in Palo Alto) and experienced no disruption today.

To totocalimero and chukra: The truth is out there.

[LauraB706]

I live in Gilroy, approx 25 miles south of San Jose. No tele, internet, cell communications. The police department personally visited every Gilroy bank, recommending their closure. At my bank in the morning, there was a large crowd outside. For security purposes, the bank only allowed 2 customers inside at a time, and the bank could only do limited transactions.
The bank was concerned that if a problem arose, they would not be able to contact the police since both the landlines and cell phones were inoperable. So by early afternoon, they closed down.
BestBuy in Gilroy closed today. Rumor had it, Costco did the same. So many people pay by credit card, and the POS terminals wouldn't work.
Other businesses closed, for security purposes - no landline & no cell means no security.
My boss was concerned that if someone broke in after hours, the police wouldn't be notified.
My friend picked up her daughter (junior high) immediately after school, since the daughter & her daughter's friends cell phones weren't working. Again, security measure.
Very spookie day today, in south county. Makes you look at things a litte differently. The things we take for granted.

[GrigoriRasputin]

People, this underscores what I have tried to tell folks for years. You should have some type of means of communicating with your friends and family in the event of a major system failure such as this. These things will happen from time to time just as a result of component breakdowns, storms, or the like. I, too, am wondering if someone was testing system redundancy with a more sinister idea in mind. This doesn't sound like the antics of a few bored high-schoolers with no supervision and nothing better to do.

Primitive though it may be, alternative communications that one might consider include ham radio, GMRS radio using the full-power radios the system was originally designed around, or for short range, CB radio or even the little FRS/GMRS radios you can find in Wal-Mart and similar places if your family works, lives, and goes to school in an area of only a mile or two. When uncertainty gnaws at your heart, having one or more of these means of contact with family and / or others can be well worth the investment.

[UpajOs]

Unless one has a prior agreement with friends, relatives and law enforcement authorities, there's no guarantee that anyone will be monitoring radio calls for help: I'd be surprised if local police or sheriff's patrols monitor CB Channel 9 at all anymore.

[shanedr]

Did you misplace your brain? Those weren't vandals, they were saboteurs! The FBI probably refers to them as terrorists; terrorism is not confined to a nationality, religion or race.

They knew exactly where to go, how to access the cables and how to sever them. Vandals would be clueless. These people acted with specific knowledge.

[GrigoriRasputin]

Hi shanedr! Exactly what I am thinking. The question now is who and with what "ultimate" purpose in mind. Is it , as one poster suggested, a disgruntled communications worker who is trying to make a statement, or something far nastier in both intent, and origin? It does bear watching!


Hi UpajO. Part of the idea I expressed, I had hoped went without saying, was that close family and / or friends who purchase such equipment would make pre-arranged decisions on channels or frequencies on which to meet if normal lines of communication were to go down. You are correct that very few, if any, police agencies monitor CB radio any more. Having a CB in each family car is better than nothing at all. Also, they are STILL great for getting road information about hazards, blockages, stoppages, etc and avoiding such. Depending on radio conditions on any given day and terrain, you can expect to normally communicate from a mile to ten or more on CB, with ranges closer to the lower end being more likely.

While it involves taking and passing a simple test that most will need to read the test question manual for, amateur or "ham" radio will probably allow for the most reliable communication with the most options for the least amount of money spent. Decent "2-meter band" (VHF) walkie-talkies and mobile radios can be had new for between $100 and $200 each. Antennas for them can be small and there are dozens of repeaters available for public use in most areas, free of charge, that allow for cars and walkie-talkies to communicate for dozens of miles, easily. There are limits on what they can be used for such as no business comms are allowed, but in a panic situation such as no phones, they could be rather handy. They are also useful in severe weather as many hams act as storm spotters, reporting weather in their area to local National Weather Service offices and receiving these reports about what is coming to or going on near their area. Again, agreeing on a frequency and alternate frequencies if that one is busy in advance are just common sense ideas. Frequently, in major communications distuptions, hams will man local police stations or fire departments and take in calls from other hams that will be passed to police or fire or EMS for help.

General Mobile Radio Service (GMRS) requires an expensive license of about $90. You it operates in the UHF bands using radios like or similar to those used by many police agencies. These radios can be very costly, but again, they can provide reliable comms within a 10 mile radius on out to thirty or more miles with use of repeaters, depending on availability in your area.

The little FRS/GMRS radios at Wal-Mart and other places run for between twenty and one hundred bucks a pair. Just make sure if buying them that they are all either Motorola or not Motorola. Moto uses a protocol in their speech processing that makes them not work as well with different brands. They are good radios when used with other Moto's but other brands are not usually compatible with them. While many of these radios have greatly exaggerated ranges such as 5, 8, 12, or even twenty something miles on the package, don't believe it. For most users, the ranges is right at one mile, give or take. The long ranges indicated are for one person being on one mountaintop and another being on another mountain top at the range given. If you have a pair of these and are trying to make your way toward a loved-one while they are headed toward you, but using a different path or route, due to obstacles encountered, extending the range of your voices to each other by one mile can be a Godsend, especially if you are on foot.

Hope this helps someone.

[GrigoriRasputin]

In next to last paragraph of my last post, above, on the second sentence, please strike the word "You" at the first of the sentence.

[AppleSuxLeo]

Jobs had it done because he is pissed about the Palm Pre ? ;)