Updated 3:15 p.m. PDT April 14 with Amazon saying the problem has been fixed and 2:15 p.m. with insider saying it was manual error by Amazon worker in France and 9:45 a.m. with background on Weev and comment from sources who say he is most likely not involved in the Amazon incident.
Amazon got blasted by gay rights groups this weekend after gay and lesbian book titles were delisted from its site. Was it an internal glitch, as Amazon claims, or is an Internet troll with a vendetta responsible?
Amazon spokeswoman Patty Smith told CNET News on Monday that the "glitch" was being fixed, but declined to elaborate. (By Tuesday afternoon the problem was all fixed, she said.)
"This is an embarrassing and ham-fisted cataloging error for a company that prides itself on offering complete selection," she wrote in an e-mail statement.
"It has been misreported that the issue was limited to Gay and Lesbian themed titles--in fact, it impacted 57,310 books in a number of broad categories such as Health, Mind and Body, Reproductive and Sexual Medicine, and Erotica," the statement said. "This problem impacted books not just in the United States but globally. It affected not just sales rank but also had the effect of removing the books from Amazon's main product search."
However, a Live Journal blogger with the alias of "weev" claims he did it to cause an outrage among the gay community, which he alleges has repeatedly flagged his online ads on Craigslist as inappropriate.
"I guess my game is up! Here's a nice piece I like to call 'how to cause moral outrage from the entire Internet in ten lines of code,'" he writes on his blog.
Weev said he figured out that he could easily get the books removed from search rankings by reporting them as inappropriate through a link at the bottom of the book page. He also claims he wrote code to identify all the gay and lesbian metadata-tagged books on Amazon and grab their IDs. He then hired people outside the U.S. to register new accounts en masse to help push the books out of the system, he said.
"Now from here it was a matter of getting a lot of people to vote for the books," he wrote. "The thing about the adult reporting function of Amazon was that it was vulnerable to something called 'cross-site request forgery.' This means if I referred someone to the URL of the successful complaint, it would resister as a complaint if they were logged in. So now it is a numbers game."
Amazon's Smith dismissed the claim and insisted the error was internal. She is not alone. Several sources have questioned Weev's account, particularly given his notoriety as an Internet troll, someone who flames others in online discussions and is intentionally disruptive on the Web.
Blogger Mike Daisey, who worked in customer support and business development at Amazon from 1998 until 2001, wrote on his blog that: "Someone was editing the category systems inside of Amazon.fr, made an error, and that system is global, so it propagated everywhere. I have no insight as to anyone's nationality, or whether it was a language gap, or anything of that nature."
Smith declined to comment on Daisey's explanation.
A Seattle Post-Intelligencer article quotes an unnamed Amazon employee who confirmed the report of manual error. "Amazon managers found that an employee who happened to work in France had filled out a field incorrectly and more than 50,000 items got flipped over to be flagged as 'adult,'" the source told the newspaper.
Blogger Bryant Durrell said he tested out Weev's concept and doesn't believe it is legitimate, partly because of buggy code.
"Summation: nope, you didn't do that, you liar you. Nice meta-troll, though," Durrell wrote on his blog.
"The really interesting thing about the troll is that he's right even if he didn't do it. The vulnerability he describes exists anywhere you make automated decisions based on third-party input."
Among the more than 1,500 products on Amazon that have been tagged "amazonfail" are "Lady Chatterley's Lover" and "Brokeback Mountain."
(Credit: Amazon)Netflix is trying to locate the source of a series of glitches that some users of the company's streaming service are blaming on Microsoft's Silverlight video player.
Over the weekend, a steady stream of angry messages was posted to Netflix's blog. The complaints range from choppy video, to audio that doesn't sync with the picture, to grainy image quality.
The complaints began accumulating soon after the Web's largest video-rental service switched to Microsoft's Silverlight in November. The posts appear to have trickled in until last weekend, when a score of customers began reporting problems.
"The quality of the video looks like bad VHS," wrote someone on the Netflix who identified themselves as Steve-O. "I use an Acer Netbook over my home network and the quality is poor. Also, I cannot even see the button to make the video full screen (using Acer One Netbook with Firefox browser). However, I imagine this will make the quality even worse. What a disaster."
Netflix spokesman Steve Swasey said the company, which now has more than 10 million subscribers, has yet to pinpoint the cause of the malfunctions. He added that only a small percentage of customers have complained but that the company continues to look for a fix.
Swasey defended Silverlight. He said the company has received mostly kudos from customers after switching to the software last fall. A Microsoft spokeswoman declined to comment.
"The new player is far better," Swasey said. "It's faster, more reliable and provides greater stability. The vast majority of Netflix members have had a great experience."
The other major complaint from Netflix customers who posted to the company blog this weekend is that it wasn't made clear there was no way to opt out once they switched to the Silverlight player.
"I certainly feel that Netflix was not forthright in getting me to 'upgrade' to the new viewer (Silverlight)," said someone who posted under the name Jerry. "I don't have a beef with Microsoft. I'll support most technologies that work appropriately--and that is where we have come to a problem. The new viewer simply does not work well enough."
Silverlight has received mixed reviews in the past year. NBC chose Silverlight to stream video of the 2008 Summer Olympics and the company won some favorable reviews. For the games' opening ceremony, Silverlight helped deliver more than 70 million page views in one evening.
But after the conclusion of the games, NBC went back to using Flash. Another setback for Microsoft came when Major League Baseball Advanced Media, the group that streams baseball games over the Web, decided to drop Silverlight.
Some owners of Microsoft's Xbox 360 who have signed up to receive movies via Netflix's streaming movie service have seen the same loss of video quality as owners of Roku's Netflix Player.
A mysterious glitch has been hobbling video streams distributed by the two set-top boxes for at least two weeks, said a source close to Netflix on Thursday. The problem is the first setback for Netflix's streaming, which has won accolades since the $99 Roku box debuted last spring. Microsoft, LG Electronics, and Samsung followed with their own set-top boxes featuring Netflix's service. Microsoft did not respond to an interview request.
Those affected by the problem have been frustrated by long delays before a movie is playable and "unwatchable," according to one Roku box owner. Steve Swasey, Netflix's spokesman confirmed the company is working on a fix but declined to discuss which set-top boxes have seen a drop-off in video quality. He said the glitch is affecting a relatively small number of Netflix users.
"We can't stress highly enough that we want everyone's experience to be the best it can be," Swasey said. "We've seen the (complaints on the message boards). It's a small number of people involved but we don't want to diminish the fact that's it important."
The malfunction has apparently stumped Netflix and Roku engineers for weeks. Roku, which counts Netflix as one of its financial backers, asked those affected by the problem to post information on the company's forums that could help the company pinpoint the problem, such as their ISP, what kind of connection speeds they typically get, and what part of the country they live in.
Earlier this week Roku posted a message on the forum asking any San Francisco Bay Area residents having problems whether they would mind letting engineers poke around their system for clues. Netflix has also been working hard on the problem, Swasey said.
"We're doing all of the analysis we can," Swasey said. "We're looking at region, at carriers...we're working diligently to identify the problem. Until we have, we certainly don't want to speculate at all. Look, there's no manual to take off the shelf here. Netflix has created something new here."
Delivering full-length movies over the Web is extremely complicated and has always been plagued with long download times or fuzzy streams. But Netflix is also competing with itself here. The streaming service was free of these kinds of quality issues for six months. Why are they cropping up now?
A Roku executive told me earlier this week that the problems coincided with changes Netflix was making with its content distribution network (CDN). It may be a coincidence but about the same time Roku customers began seeing problems with video quality, some Apple TV owners began complaining on the company's message board about long delays downloading movies.
Netflix suffered a nearly hour long site outage on Monday, the company confirmed.
The site went down at about 3:40 p.m. PT and Netflix posted this note to the site's front door at about 4 p.m.:
"We're sorry, the Netflix website is temporarily unavailable. Our shipping centers are continuing to send and receive DVDs , so your movies will be processed as usual. And you can still instantly watch movies via your Netflix ready device."
Netflix has suffered several costly outages in the past year. A hardware glitch prevented the company from shipping DVDs to customers for several days in August. A malfunction at the company in March also delayed shipments.
In what is likely an unrelated event, some users of the Netflix Player by Roku have complained for two weeks of receiving dramatically inferior video quality in the movies Netflix streams via the Roku box. Roku executives told me they suspect the source of the problem is Netflix's content distribution network (CDN). Netflix's Swasey said the company is investigating.
Roku's November 13 note to customers asking them for help determining the cause of a dramatic decline in video quality.
(Credit: Roku)The Netflix Player by Roku, which enables owners to watch streaming Web video on their TV sets, has received plenty of applause from pundits and owners since debuting last May. But the box now faces its first major challenge.
Customers from around the country have been "experiencing inexplicable loss of video-streaming quality," for at least three weeks according to Roku's engineers, who have posted comments at the company's Web forums. Device owners have posted complaints to the same forums about receiving less than half of the video quality they've had in the past. One user told CNET that the video stream is now "unwatchable."
Tim Twerdahl, vice president of consumer products at Roku, told CNET News on Monday that the company is still unsure about what exactly triggered the problem, but he said indications are it originated at Netflix. Twerdahl added that the problem likely affects Netflix's other boxes as well as Roku's player.
"All we know is Roku didn't make any changes," Twerdahl said. "This is not a box problem. We know from some reports that this seems to be correlated with a change in Netflix's content distribution network (CDN), and Netflix is trying to figure out what the issue is."
... Read more
Updated at 3 p.m. PDT with details from the FAA.
Hundreds of flights were delayed in cities across the country Tuesday because of a computer failure in the Federal Aviation Administration's system for processing flight plans.
A representative from the FAA said a software problem in the administration's central system for processing flight plans, based in an Atlanta office, caused the system to go down at about 1 p.m. EDT on Tuesday. That failure prompted a backup system in Salt Lake City to take over flight-plan processing, but a backlog in the handover caused flight delays instead, according to FAA officials.
"There were about 5,000 flight plans in the system, but a lot of them were airborne and unaffected," said Hank Krakowski, chief operating officer of the FAA's Air Traffic Organization. "The only flights that were affected were those that had already pushed off from the gates and couldn't get off the ground. It created a backlog."
The FAA said it expects the problem to be fixed by about 6:30 p.m. EDT.
The problem largely hit cities in the Midwest and on the East Coast, but the FAA was unable to be specific about how many airports or flights were affected. Those airports hit hardest by delays were in Boston, Atlanta, and the Washington, D.C. area. Washington National was still experiencing delays as of 6 p.m. EDT. Chicago O'Hare had experienced 60- to 80-minute delays throughout the afternoon. And Atlanta had as many as 40 aircraft backed up during the afternoon, according to FAA officials.
To put the software trouble in perspective, a bad day of thunderstorms might cause more flight delays than those experienced on Tuesday. But the FAA's computer issues likely had a larger geographical affect, officials said. Part of the work in diagnosing the computer failure will be in understanding why some airports were more affected than others, officials said.
"There were so many aircraft and flight plans...it overwhelmed the system. It rejected plans and increased delays, and added to the volume, so we're managing our way out of it," Krakowski said.
At the heart of the problem was the FAA's computer system known as NADIN, or National Data Interchange Network, which processes the flight plans that airlines file every day. The FAA always runs a parallel system in the event of a software glitch, but officials said that they had never encountered the challenge they had Tuesday.
"This was a failure mode we've not seen before.," Krakowski said, without describing the problem exactly. He added that the agency did not suspect any hacking or safety issues.
"It looks like an internal software processing error. We think we know what it is, but we have to do forensics on it to figure it out," he said.
By the end of this year, the FAA plans to upgrade the NADIN software. Representatives did not say how the software will improve on the current system, however.
Netflix has traced the causes of a lengthy system outage this month that prevented the online movie rental service from shipping for several days to a hardware glitch.
The good news for the company is that it received lots of help from vendors to determine that the cause of the outage, which hobbled the company's ability to ship DVDs from August 11 to August 15, was a "key faulty hardware component." Steve Swasey, a Netflix spokesman, said the company's "strongest aspiration is to safeguard this from ever happening again."
The bad news for the company is that Netflix can't yet prevent these system outages. While a piece of hardware, about which Netflix declined to provide details, was the cause of the latest glitch, it was not responsible for the system crash back in March, Swasey confirmed.
In that earlier case, a malfunction knocked out the company's Web site, as well as its logistics and delivery systems, for 12 hours, the company then said. In both cases, Netflix was unable to ship movies "to a large number of Netflix customers."
Two major meltdowns in 2008 raises serious questions about the soundness of Netflix's system. Tony Wible, an analyst at Citigroup, estimated that Netflix lost $1.8 million to $3.6 million for each of the days it was down. The 15 percent credit Netflix is providing to affected customers will reduce the company's third-quarter revenue by $6 million, according to Michael Pachter, an analyst at Wedbush Morgan Securities.
As more and more competitors jump into Web video rental, annoying system blackouts could become more of a liability for Netflix.
In both outages, Swasey said many customers hardly noticed the delays. Many users who were expecting packages weren't put out very much by waiting a day or two extra for their films. Customers never lost money or personal information, and by most anecdotal accounts, the service remains extremely popular with users.
At the root of the problem is the fact that Netflix is quite different from most e-commerce companies. It relies on the Internet, the U.S. Postal Service, and a groundbreaking fulfillment operation that combines software, hardware, and plain old elbow grease to ship those little red packages.
Netflix has won accolades for using technology to wrest market share away from big brick-and-mortar video renters, such as Blockbuster. The company has amassed 8.4 million customers and ships more than 2 million movies per day.
But perhaps now is the time that Netflix would do well to invest in a major system upgrade. If Netflix customers continue to see delays, it could undermine the company's credibility. Does Netflix really want users to consider Blockbuster's in-store kiosks? I don't think so.
- prev
- 1
- next
