Digital Media

Weeks ago, the news of Michael Jackson's passing brought major news sites to their knees, so Tuesday's memorial service for the singer was expected to bring similar results.

This time it appears sites were better prepared for the traffic onslaught.

According to Gomez Incorporated, a company that monitors Web usage quality, there were both slowdowns and outages, including one that dramatically slowed Twitter's performance. The company analyzed performance on seven news sites from multiple locations during Tuesday's event, with some of the biggest slowdowns coming to streaming video. Asia experienced a 40 percent increase in what the company calls "stalling issues," with the U.S. experiencing an increase of around 5 percent.

One of those news outlets that was serving up live streaming video was CNN, which according to internal data, topped out at 781,000 concurrent streams of the event. Between midnight EDT and 4 p.m. the site also pulled in 11 million unique users who turned 72 million pages.

Ustream, which provided live streaming in a partnership with CBS, says the event was the "largest ever" that had been hosted on the service, in part because it was a worldwide broadcast. The service had 4.6 million streams of the memorial going, made up from 1.6 million unique users. It also had more than 12,000 messages posted every minute to its built-in user chat rooms. (CNET News is published by CBS Interactive, a unit of CBS.)

Besides slowdowns in streaming video, news sites also had lower availability, which means some users were unable to access them. Gomez recorded that number as low as 98.2 percent, whereas the sites usually maintain uptime in excess of 99.65 percent. Response times also took a hit. News sites experienced double, and nearly triple the load time to serve up pages. In the case of Twitter, many users were unable to view or post messages to the service. At what was seemingly the peak of Twitter's load, Gomez benchmarked it as taking around 62 seconds for the site's home page to load, then allow users to log in--a process that normally takes just a few seconds.

Update: See also Larry Dignan's analysis over at ZDNet. He points to data host Akamai's visualization tool, which shows real-time activity on its sites which represent around 20 percent of the Web's traffic. There's a noticeable bump around the time the memorial service begins.

Internet Web traffic hit its peak right around the beginning of the service, according to Akamai.

(Credit: CNET / Akamai)

CNET News' Greg Sandoval contributed to this report.

This story was updated multiple times after it was originally published, including with a Keynote System statement that it erred in assessing the performance of ABCNews.com and other media news sites.

Click for full screenshot

(Credit: CNET News)

It turns out many of the Internet's top news sites fared better at handling the glut of traffic following the death of singer Michael Jackson than previously thought.

Keynote Systems, a company that tracks site performance, said Friday that it erred in measuring performance for news sites and issued incorrect information Thursday evening.

Dan Berkowitz, Keynote's spokesman, said the most important thing was to correct the record about ABCNews.com, which he said delivered pages to visitors close to normally during the hour-long traffic spike following Jackson's death. He added that the company is still studying the causes of the error. A representative for Disney, which owns ABC, said the company saw "no dips in performance" as a result of the traffic glut.

While Keynote's assessment of site performance may have been overly bleak, many news sites were slow to deliver pages and in some cases were inaccessible at times.

When news of the iconic performer's death began trickling out, scores of people turned to the Web for information. TMZ broke the news that Jackson, 50, known for producing some of the world's best selling records, including "Thriller" and "Bad," had died Thursday afternoon, but the gossip hub cited only unnamed sources and offered few details. As other news services turned their attention to the story and as the public took to the Web to learn more about the performer's condition, some Web sites began slowing down.

Some Google users complained that the search engine's News area was inaccessible for a time.

A Google representative confirmed that "between approximately 2:40 p.m. PDT and 3:15 p.m. PDT today, some Google News users experienced difficulty accessing search results for queries related to Michael Jackson."

CNN.com appeared to be sluggish delivering Jackson stories at times. In its defense, the news organization said that the site saw 20 million page views and a fivefold increase in traffic (from where it was prior to when news about Jackson's death began widely circulating) in one hour.

Even before Jackson's death, Thursday was a big day for news sites as word of actress Farrah Fawcett's death hit the wires in the morning and continued interest in the scandal surrounding South Carolina Gov. Mark Sanford.

The traffic deluge came swiftly and lasted for about a half hour, according to internal data here at CNET News, which saw twice the normal amount of hourly traffic shortly after word of Jackson's death spread. At sister site CBSNews.com, traffic numbers were five times their normal levels.

On Friday, antivirus vendor Sophos reported on a wave of spam related to Jackson's death that claims to have vital information about the news event. There are no malicious URLs in the spam, but recipients who reply to the message are then providing proof that the e-mail address is legitimate and will likely be targeted in future spam campaigns, the company said.

Corrected on June 26 at 4:07 p.m. PT: Keynote Systems, the source for Web site performance supplied CNET News with incorrect data. ABCNews.com's performance following the death of Michael Jackson was near optimal.

Michael Jackson's death sent U.S. Internet users to news sites and the traffic heated up network traffic. as evidence in these images.

(Credit: Akamai)

Updated 10:57 a.m. PDT with comment from MLB.com.

When is 2 million people too small a group to get a good feel for the latest trends?

When you're trying to perform the deceptively difficult calculation of just how many people visited a Web site.

ComScore, by some accounts the leading analysis firm that supplies publishers and advertisers with figures about how many people visit various Web sites, announced Tuesday a switch to a new method geared to provide more accurate Web traffic numbers. Previously the company had judged traffic on the behavior of a panel of 2 million persons scattered across the globe, but the new service, Media Metrix 360, combines that data with statistics taken directly from the Web servers that can tally visitor totals.

The hybrid approach is a big departure for ComScore, which for years has staunchly defended the panel approach. ComScore Chief Executive Magid Abraham expects the move will put to rest most complaints by publishers that ComScore's statistics dramatically underestimated real popularity.

"There is a big gap that we will rectify," Abraham said. "My personal belief and hope is it will address 90 percent-plus of the issues...Unless there is a widely different method for measurement that a publisher is using on their own, we should not really see differences anymore."

Traffic discrepancy
The hybrid approach still is fundamentally panel-based, but uses server data from participating publishers to inform the totals. ComScore expects it will estimate traffic better from computers that panels miss today: mobile phones, larger companies, cybercafes in Asia and Latin America, and public terminals at schools and libraries, Abraham said.

Measuring traffic is a long-running point of contention between publishers and ComScore; MySpace and Major League Baseball's MLB.com are among those who've objected in the past to panel-based numbers.

Torstar Digital, a division of the parent company that owns the Toronto Star, is one company that's had trouble reconciling its newspaper sites' numbers with ComScore when it came to tallying how many times visitors viewed pages, said President Tomer Strolight.

"We regularly saw discrepancies between ComScore page views and unique visitors of 3 to 1 or greater when compared with our server-based tools on those sites," Strolight said. ComScore's panel-based approach posed problems when it came to extrapolating statistics from people at work and older audiences, he said, but the new service helps.

"The discrepancy, and that ratio in particular, are not present in all sites by any means, but it does happen to affect my largest site and it is therefore very important to me that we resolve that issue," Strolight said. "ComScore's new methodology can do this."

MLB.com CEO Bob Bowman gives credit to ComScore for moving beyond just panels, but he's still skeptical, in part because of what panels miss from work users. Instead, he'd prefer to share his server log files after they'd been vetted by a neutral party.

"For our site, 70 percent of day traffic comes during working hours. Missing that is comparable to saying we built a beautiful boat--it just doesn't have a bottom yet...I reject completely that panels can ever work when it comes to what people do at work," Bowman said. "I don't know why for top-250 sites, such as MLB.com, our files just can't be audited, and (the auditors) say yup, here's the traffic."

So why does the number matter beyond bragging rights in a press release?

Money, of course. Advertisers want to know if they're showing an ad to the same person multiple times or to different people. And of course Web publishers want to know how many people really do use their site.

"A media market develops on the basis of trusted information for all the participants. The sellers and buyers have to agree the numbers are something they can live with," Abraham said.

Not as easy as 1-2-3
One might think it a simple matter to measure how much traffic a Web site gets. Just keep a log of the Internet addresses of visitors, or perhaps deliver the "cookie" text file to their browser for easier identification of repeat visitors, right?

Wrong. There's often a discrepancy between independent panel-based statistics from companies such as ComScore or competitor Nielsen Online on the one hand and server-based statistics from a Web publisher's internal logs or third-party services such Google Analytics or Omniture on the other. Here are some factors that can inflate Web site visitor statistics based on server logs:

• The same person might visit the same site from work, home, and increasingly, from a mobile phone. That's not a problem when counting total traffic to a site, but it is when trying to tally unique users.

• Sometimes people delete cookies either manually or automatically through antispyware software, meaning that a cookie might be delivered to a person who seems to be a new user but who in fact has visited a site before.

• Someone might visit the same site with multiple Web browsers or open a tab in a browser without actually making it active.

• Computer servers such as search engine indexers can visit Web sites.

These issues are diminished when users must log into a site, making it easier to track individual use, but the panel approach attempts to address the issue more broadly, consistently, and independently. Panel-based information also answers a question that an individual site cannot: how often is a particular person exposed to the same ad while browsing multiple sites on the Web?

"Ultimately, we need to report unique people, not unique machines, unique cookies, or unique browsers," Abraham said. "There is a lot of energy that goes on trying to reconcile the numbers and trying to explain to people the ins and outs and the subtleties of why this number is not that number."

Panel shortcomings
However, panel-based measurements have their own shortcomings, in part because they rely on software installed on users' machines. Thus the difficulties with mobile phones, businesses, cybercafes, libraries, and schools, Abraham said.

Cybercafes are widely used in Asia and Latin America, he said. Mobile usage for typical sites accounts for less than 1 percent of traffic today, but it's much larger--potentially more than 20 percent--for sites that appeal to mobile users such as those handling weather, stock quotes, breaking news, sports scores, local information, and social networking. And today, ComScore largely just estimates traffic from big-business users.

"It's really difficult to recruit users to participate in panels in large corporations," Abraham said. "Large businesses are in essence voted for by the medium-sized businesses, by proxy. Sometime that works, sometimes that doesn't. That's one area of improvement this (Media Metrix 360) will create."

The company has begun offering panel software to some mobile users but doesn't yet publish resulting data. "We do have that developed for a number of smartphone platforms such as Windows, Palm, and (BlackBerry maker) Research In Motion. We are working on solutions for iPhone and Android," but it's difficult to deal with the plethora of models in the market, he said.

ComScore recruits panel members by offering them free software such as games and screensavers and through incentives including sweepstakes and, more recently, an offer to plant trees in third-world countries. The panel size of 2 million people spans 170 countries, enough for global estimates and for specific measurements in 40 countries. The company also uses technology that can distinguish different users on the same machine by identifying signature patterns in mouse and keyboard use, an important factor for shared computers.

The switch to the hybrid methodology will be gradual. Publishers must add a transparent pixel to their Web sites that ComScore uses to track visitors, Abraham said, and participating sites undergo a 60-day "incubation period" to make sure data collection is working and nobody is gaming the system, he added.

"We think we'll get widespread support. There is widespread hunger for this," Abraham said. "From many people we've heard, 'What took you so long?' It's a fair question. The answer to that is it's not as easy as it first appears."

Immediately following the enactment of a new Swedish antipiracy law on April 1, Internet traffic in Sweden plummeted--and it has yet to return to prior levels.

According to Netnod, an organization that measures Internet traffic on access points between Swedish and international networks, traffic went down from average data speeds of about 160 gigabits per second to about 90Gbps and has remained so since the day the new law went into effect.

On April 1, in the middle of week 14, the new antipiracy law took effect in Sweden.

(Credit: Netnod)

Netnod has declined to make the connection between the new antipiracy law and the traffic drop since it only measures traffic without identifying what sort of activity is behind the numbers. Other large Internet service providers won't release their numbers.

But Jon Karlung, CEO of Bahnhof , a comparatively small, outspoken broadband operator that has expressed opposition to the new antipiracy law, explains what it has seen.

"Almost half the Internet is gone," Karlung told CNET News over the telephone from Sweden. "Likely, it is the torrent traffic that has declined, but I cannot say whether this traffic is legal or illegal."

The so-called IPRED originated from the European Union's "International Property Rights Enforcement Directive." IPRED stipulates that property rights holders can take their grievances to a court, which will examine the evidence and decide whether the name of a holder of an IP address will be released.

The guilty verdict in the high-profile Pirate Bay trial, announced earlier Friday, was not affected by IPRED, since only file sharing done after April 1 is being affected by the new law. But copyright holders have already turned to the new law in an attempt to stop file sharers.

On the law's first day, five Swedish audio book publishers went after an alleged illegal file sharer in court, in hopes of revealing the identity of the person behind a particular IP address.

And two days after the law came into force, two men were arrested, allegedly for sharing copyrighted files and administering a "rip box," which removes copy protection on purchased films and music. International police were involved in the arrests.

Jan Karling, CEO of ISP Bahnhof will not store traffic data of alleged pirates.

(Credit: Bahnhof)

But now Bahnhof says it won't release the names connected to IP addresses, since its understanding of an earlier law based on another EU directive is that ISPs must erase traffic data for the sake of the subscribers' integrity.

"Our ambition is not to store any traffic data," Karlung said, adding that as a consequence, "Bahnhof cannot provide information on alleged piracy to courts, since we do not have the information stored. Thus IPRED becomes effectless."

Bahnhof's interpretation of the earlier law gets support from the Swedish Post and Telecom Agency, a regulatory body that's akin to the Federal Communications Commission in the U.S.

"There is no general obligation to store this kind of data for all subscribers," PTS attorney Peder Cristvall told the magazine Computer Sweden.

Bahnhof says it's opposing the new antipiracy law since it stops Internet innovation and development, naming Swedish companies MySQL Skype , and Spotify as examples of companies whose success has benefitted from Sweden's extensive file-sharing culture.

Instead, Bahnhof says copyright holders must develop business models and Internet tools that allow subscribers to share files legally.

Karlung says that in the short term Bahnhof's profits will rise with the IPRED law due to lower bandwidth costs, but in the long term the sales of fast Internet connections used for file sharing could decline.

"It is possible that we and other ISPs could sell fewer fast connections, but that won't affect our profits," Karlung said.

But just to make things a bit more complicated, the Swedish government is expected to propose a new data storage law based on a third EU directive in June. This law could force Bahnhof to store and share its data in the future anyway, much to Karlung's disappointment.

"It is this Orwellian nightmare state that is developing, where no one sees the dynamic of the Internet," a sighing Karlung says from the other side of the Atlantic.

So either Jon Stewart is really on to something with his mad-as-hell crusade against financial hypocrisy and stupidity, or there are a lot of unemployed people watching Comedy Central clips to pass the time.

Either way, an on-air freakout by CNBC reporter Rick Santelli may have been one of the best things to happen to Comedy Central in months: Fake-news pundits Stewart (of "The Daily Show with Jon Stewart") and Stephen Colbert (of "The Colbert Report") have seen traffic to their Web sites and online video clips soar after the two went on mocking vendettas against Santelli, fellow CNBC personality Jim Cramer, and the NBC Universal-owned business network in general.

Traffic to the shows' Web sites has been at its highest of the year so far in the past week, at over 60 percent their weekly average for 2009. ComedyCentral.com, which hosts video clips of both programs, also had its best traffic of the year, and the digital version of a viciously funny clip called "CNBC Gives Financial Advice" logged over 1.3 million views in a week, the sort of numbers usually reserved for grainy videos of cats behaving unnaturally.

Here's the back story: Santelli was supposed to appear on "The Daily Show" after his tirade about the federal government's economic bailout, but backed out abruptly. That's when Stewart and Colbert--but especially Stewart--turned up the heat. Stewart went on the aforementioned anti-CNBC rant on March 5, putting "Mad Money" host Jim Cramer squarely in his crosshairs. Cramer appeared on "The Colbert Report" the following night.

Now, Cramer is scheduled to make a "Daily Show" appearance on Thursday night.

Stewart and Colbert have been two of the most visible figures in cable television's slow crawl onto the Web. Not only are they wildly popular with young and tech-savvy audiences, but the segmented format of their talk shows lends itself well to being split into short clips and swapped via video-sharing sites, which meant that unauthorized clips of the two were some of YouTube's earliest hits. That's what indirectly led to Comedy Central parent company Viacom's massive copyright lawsuit against YouTube owner Google.

Later on, the full archives of both shows were made available on Comedy Central's Web site, and recent episodes are available in full on Hulu (as well as iTunes and Xbox Live).

Colbert, who started out as a commentator on "The Daily Show" before spinning off his blowhard persona into his own talk show, also owes a big chunk of his notoriety to the Web. Video of C-SPAN's coverage of the White House Press Correspondents' dinner three years ago, in which Colbert performed a shockingly blunt comedy routine that skewered then-President George W. Bush, was a huge hit on the Web among those who wouldn't have considered actually watching C-SPAN in the first place.

Last year, Colbert was honored by the annual Webby Awards as "Person of the Year." Take that, nonbelievers!

(Credit: Akamai)

Web usage spiked on Thursday as people looked for news of the U.S. Airways jet that went down in New York's Hudson River.

Shortly before 5 p.m. EST, Akamai, which has assembled a content delivery network used by many global news organizations such as CNN, CBS, NBC, Reuters, and the BBC, reported a huge spike in Internet usage as people looked for news and video of the event.

The crash now ranks as the seventh biggest Internet news event since Akamai started tracking spikes in traffic in 2005. The plane crash, which miraculously resulted in no fatalities or serious injuries, ranks just ahead of the post-Election Day 2008 coverage.

Election night 2008, when Barack Obama was elected the 44th president of the United States, had the biggest spike in Internet usage. Over 8 million visitors per minute went online on November 4, Akamai's data shows. About 4.9 million visitors per minute were clicking on Web sites after the U.S. Airways crash landing.

Akamai is expecting a record-breaking spike in Net traffic on Tuesday when President-elect Obama becomes the first African-American to be president of the U.S.

Of course, my colleague Charles Cooper thinks all this hoopla made over keeping score of which news events elicit the most activity in the virtual world is silly. Check out his contrarian view of these virtual reactions in his latest column.

Daily market share in "All Categories" as measured by visits, based on daily usage.

(Credit: Hitwise)

Did we turn to Facebook for Christmas cheer, or to cheer up Christmas?

The social-networking site, which has been experiencing explosive growth in membership, saw record traffic on Christmas Eve. Facebook achieved its highest-ever traffic level, accounting for 2.18 percent compared with a 1.42 percent average for November, according to numbers gathered by Hitwise. That's a 54 percent increase compared with the November average and a 53 percent increase year over year.

That pattern was mirrored in the U.K., where visits to the social networking site had a market share of 4.65 percent, accounting for one in every 22 Internet visits.

So, in this season when faithful friends gather near to us, are we substituting Facebook for face time with our loved ones?

Hitwise's Heather Hopkins offers some theories in a blog post on what might have caused the holiday traffic spike.

Facebook's top markets of New York, Chicago, Washington, Boston, and Philadelphia were all hard hit last week by severe weather, which may have prevented many people from getting out and visiting in person, Hopkins notes.

Noting that Christmas Day was Facebook's busiest traffic day in 2007 (one day later than 2008), Hopkins suggests that boredom--when coupled with the weather--may have contributed to the increase.

"I received 5 friend requests last week and many holiday wishes," Hopkins writes. "Maybe people were simply bored while stuck home with family and so escaped to computers to catch up with friends."

Perhaps the best explanation is that more people were using the site to send their late holiday greetings. Hitwise also saw increases in traffic at Yahoo Mail and e-greetings Web sites, Hopkins said.

Whatever the reason is for the record traffic, a better question is whether its revenue can keep pace.

The social network put out stats last month that peg its active-user count at 140 million. But Facebook's growth is primarily overseas now, and its international pull is responsible for those skyrocketing numbers.

Especially overseas, server power can be costly. Facebook has raised a ton of venture capital, is reportedly hunting for more, and says it's in good financial shape. That comes back into question, however, if it's growing faster than it ever expected to.

When consulting online traffic maps to form your plan of attack for hitting the streets, how often do you suspect that the red, yellow, and green colors indicating the various speeds of traffic flow are inaccurate, show outdated data, or that they'll change by the time you get there?

The concept of online traffic maps makes a lot of sense, but until they're foolproof, users will always be skeptical. A new collaborative project between UC Berkeley and Nokia is trying to provide mapped traffic data with more accuracy than ever before. How? By tapping into the ubiquity of GPS-enabled cell phones and the willingness of drivers like you to share your location information.

Here's how the pilot project, called Mobile Millennium, will work. Volunteers with phones running on T-Mobile or AT&T's services can register their phones and download the appropriate software through the pilot's Web site. Nokia does not need to be the manufacturer of the hardware, but the phone obviously needs GPS and has to be able to run Java applications, like Blackberries and iPhones. And yes, right now, you do need to live in the Bay Area.

Registration is free and takes less than five minutes. At this point, your work is essentially done. As you drive with your phone in the car, you'll cross the virtual trip lines placed every quarter mile on the NAVTEQ maps on the program. When you do, your phone knows to send its coordinates and traveling speed back to the engineers at UC Berkeley, who have created the algorithms to process the data.

This kind of program will only work where there's a large enough sample size to analyze, and that may not happen for a several months or even years. The advantage of this program, however, is that people with cell phones drive on roads that traffic cameras and actual trip lines simply cannot access to provide travel information. Imagine having access to traffic stats for city streets, rural roads, and vacation routes--and not just commuter thoroughfares.

(Credit: Hitwise)

Reflecting consumer pessimism toward the current economic climate, online retailers experienced a decline in visits for the eighth straight week, according to a report released Tuesday.

Traffic to a custom collection of 500 retail Web sites monitored by analyst Hitwise declined 3 percent for the week ending October 25 compared with the same week a year ago.

The declines mirror the overall decline in consumer spending the past couple of months. Visits to Hitwise's Retail 500 Index were up 14 percent from June through August compared with the same period in 2007, but then declined 4 percent in September.

"These declines have strong implications for the upcoming online holiday season as well as offline sales," Heather Dougherty, research director at Hitwise, said in a statement. "Everyone is aware of the role that the Internet plays to influence offline sales through research, so this slowdown may indicate a further ripple effect in sales in retail locations."

Online music retailers saw the greatest year-over-year traffic decline for the week of October 25, dropping 21 percent. Computers saw the second greatest decline at 18 percent, followed by ticketing at 12 percent. Toys and hobbies saw a decline of 10 percent, as did video and games, according to Hitwise.