Radiohead calls ComScore report inaccurate

Radiohead stuck up for its fans on Friday.

The rock band denied that 62 percent of those who downloaded the group's new album paid nothing for the music.

Last month, Radiohead announced that it was releasing a digital version of the album for whatever fans wanted to pay. Internet research group, ComScore, on Monday released a report that said only 38 percent paid anything for In Rainbows.

In a statement, Radiohead's representatives called ComScore's report "wholly inaccurate."

Radiohead's pay-what-you-want offer is groundbreaking and is being watched closely by fans, music labels and other bands. How it fares could influence whether other acts try and sell their own music via the Web--without the support of the labels.

Andrew Lipsman, a ComScore senior analyst, didn't back down. In a blog posted to the company's site on Thursday, Lipsman said that he was "very confident" in the data.

ComScore derived its numbers by watching the Internet behavior of nearly 1,000 people. Several hundred among this group downloaded Radiohead's album.

In the blog, Lipsman said that when it comes to statistics, this is considered a large sample.

"We observed the actual online spending behavior from a robust sample of hundreds of individuals in order to produce an accurate estimate," Lipsman said in his post. "If we didn't have a reasonable sample from which to extrapolate, we wouldn't have released the data."

But in their statement, Radiohead's handlers said that ComScore's study "in no way reflected definitive market intelligence or, indeed, the true success of the project."

Radiohead has declined to reveal any sales figures.

[rdupuy11]

ya well

As severely as I have criticized the ComScore report.... (and keep in mind, this article says 'more to come'..so I'm commenting on an article that will be largely changed...)...

I do respect ComScore for one thing. They gave their best numbers. RadioHead just saying they are inaccurate, but declining to give any numbers, still leaves ComScore with the most accurate numbers available.

And its a legal trick to say something that is 1% off is completely inaccurate....inaccurate is not accurate.

I have great respect for what RadioHead is doing, but no respect at all for this decision to not share the knowledge.

[groink_hi]

"whatever fans wanted to pay"....

Is this the digital version of the starving musician? Rather than hanging around on the streets with an opened guitar case, they're now opening commerce sites?

[aSiriusTHoTH]

Radio Head a starving musician? LOL

Wow.. your very out of touch. Radio Head has a huge multi-million people fan base. Not sure how you can say they are like people on the streets with open guitars. These guys routinely make millions of dollars on their albums. Think before you speak...

[Neotrope]

Reminds me of the Neilsen iTunes Store Report

The problem with so-called "industry statistics" is that they basically garner media attention for whatever company does the polling, as in this instance comScore, but may not be based in actuality, only those polled.

I recall a story earlier this year that reported Apple's iTunes store downloads were way down, based on a sampling of less than 1,000 people who had not bought as many songs online as previously that prior quarter.

As example of how that report was irrelevant... it basically said "Fewer people are downloading music from iTunes..." - when in reality all it meant was "the people we talked to are downloading less thant before..." However it didn't take into account the 30 million new users of the iPod who were potentially downloading tunes, ot the other millions and MILLIONS of people who have iPods.

Similarly, if 100,000 people download music and 15% pay something, but then a "survey" of some random 1,000 people finds only 2% of THAT sample group paid, issunig an "industry research" study saying "only 2% of fans pay for online music" is simply ridiculous and not worth the bytes or bits they're bitten on.

I generally consider such "reports" to be worthless except in understanding the metrics of the people spoken to. Polling is an imprecise measurement of anything except the people spoken to, not an entire group when the sampled amount is so small.

I am however very interested to see the "actual" report from the first 90 days of the experiment, based on "hard data" not a third party noisemaker like comScore.

(FYI, I've been a music and technology journalist for 24 years.)

[faust]

Back to school

You need to learn about sampling size and projections.

[sethum1]

Exactly

That ComScore's "several hundred" person sample could ever accurately reflect the 1.2 million users who downloaded Radiohead's album is completely ludicrous. The problem is due to the polling company's obtuse assumption that their sample is representative of the internet - which is extremely unlikely. Not only are internet users vastly different in their surfing habits, but various web services also more strongly attract certain demographics than others, creating a highly complex and clustered topography.

ComScore's sample is like giving a voluntary poll to the audience at a Britney Spears concert about whether the US should get involved with Pakistan's application of a state of emergency. Weak correlation to the rest of the country.

[faust]

Radiohead Needs to Say What Their Numbers Are

Don't come out and call a story false and then not offer up your version of the figures, you just look silly doing that.

If comScore is wrong, how wrong are they? Were they off by a lot or just a little?

[protagonistic]

RE: Radiohead Needs to Say What Their Numbers Are

Radiohead needs to say no more than they wish to about the
report. That is called the right to privacy. All they have to do is
say the report is inaccurate if that is the case and they have
done so.

And unless the people doing the sampling are illegally accessing
private data all they can say is we feel we are correct, but they
can't really prove that. If they asked me if I paid for it I would
probably either say nothing or lie to them because I would feel it
was none of their business.

[Neotrope]

How can it be accurate?

Radiohead doesn't need to specify the numbers being wrong ... since comScore is not in a position to have any of the actual data, it's only a guess based on a sampling.

Wrong is wrong, no matter how one jiggers the numbers. All stats can be positioned to prove any point, on any side of any argument (as has long been known by those of us who are in the ad industry professionally).

comScore info is false since it's not based on actual numbers, only a survey.

Surveys are not imperical data.

[boopiejones]

time will tell how successful radiohead was

my guess is that they made a boatload of money. not only that, but they definitely opened their music up to a larger audience. who would pay $10 to $15 for the album of a band they are not very familiar with? no one. but maybe some of those same people will download it for free, become fans, and then pay for the next album.

we may never know how successful radiohead was in this experiment, but if they have this same strategy on subsequent albums i would argue that it is working for them and they are making good money, comparable to or better than that of a traditional album release.

[bedelman--2008]

Questioning ComScore's Methodology & Representativeness

ComScore's analysis relies heavily on the assumption that ComScore users are like all other users. (Read ComScore's blog entry for details, including their random survey / lunch money analogy.) But let's think about where ComScore gets its data. ComScore's software automatically records what proportion of ComScore users downloaded the Radiohead album at issue. Then ComScore asks the public to believe that ComScore's users are just like ordinary Internet users -- that whatever proportion of ComScore users made a contribution, the same proportion holds true in the general population.

Is it true? I doubt it. What kind of users install ComScore's monitoring software? I have repeatedly observed the software being installed through security exploits, without user consent. (See www.benedelman.org for details.) Users who suffer from such a security exploit tend to have older computers -- hence probably less money (with which to buy new PCs), and less inclination to make a contribution to support a download. ComScore says its sample is "representative", but it's hard to be sure. What sophisticated, wealthy, expert user is going to install tracking software that records their every move? It just doesn't make sense.

ComScore's data may be a reasonable start, and maybe it's the best the public can get. (Radiohead itself knows better, of course.) But we'd be crazy to think ComScore's method is as infallible as the company would have us believe.

[Leria]

If this company's software is spyware

Then that automatically throws doubt on their numbers, like you said. If this software is usually installed through security exploits..... that puts even MORE doubt on their numbers.

I'll be blunt - ComScore should just shut the heck up, because they are in no way representative of the internet users as a whole.

[talker29]

Yes... I'm someone who didn't pay

A lot of this discussion assumes every one is a Radiohead superfan. I'm not a Radiohead fan, I don't own any of their old CDs, and have never bought any of their music from itunes. I actually haven't paid for any music in at least five years.

I downloaded their album once at home and once at work, without paying, just because I could. Listened to it a few times, thought it was ok, I probably won't listen to it that much any more. Did they lose any money? doubt it, i wasn't going to buy it anyway.

Some things to think about...
How many of those 62% weren't going to buy the album anyway and just got it because it was free? Lots of people aren't fans and were just checking it out.

How many people downloaded it more than once? Even if you paid something you aren't going to pay more to download it to another computer.

Of all the people who never listened to radiohead before and were just checking it out to see what they're all about, how many how many new fans did they pick up? that's worth something

What if you downloaded it for free just to check it out, liked it and felt it was worth paying for, and then downloaded it again so you could give them money?

If they make $1 from a CD sold at a store, one person giving $10 is worth 10 CD sales(actual nubers vary but its still a significant difference)

If you take all these things into account, I suspect they did pretty good.

How many people actually paid isn't really relevant. Its how many total downloads there were compared to previous album sales, and how much money they ended up with in their pockets compared to their last albums.

[BCF1968]

You got it wrong

"How many people actually paid isn't really relevant. Its how many total downloads there were compared to previous album sales"

Sorry you can't compare FREE downloads to actual sales.

If a band sold 5 million albums at $15 a pop but their next album was downloaded 15 million times, but was free, you can't say the 2nd album was more successful. Who's to say if the first album was a free download it wouldn't have had 20 million downloads? Or who's to say if the 2nd album was $15 that it would only have sold 2 million units?

[mirrordash]

But, was it WORTH paying for?

I was one of those who didn't pay. Mainly because I wanted to test the album for quality and was curious what bit rate it could be downloaded at.

If I liked the album, I was going to pay for a second download (since that would be the only way to pay later). As it turned out, the album was very lackluster and uninspiring for what I have come to expect from Radiohead. So, it failed that part of the tests.

The bitrate test also failed. I thought Radiohead was a band OBSESSED with sound quality, and for them to offer dowloads at 160 bit rate, is surprising and dissapointing. I would expect near lossless (or lossless) compression. For anything less, I would only be willing to spend a minimal amount. As an example, a 10 track album downloaded from eMusic costs about $2.50. That's about the most (and I still think too much) I'm willing to pay for a non-CD quality album.

Now, if all of these companies would start allowing downloads of CD quality audio, it might be worth it.

[D.A.W.]

I agree except for one point

I did exactly the same thing you did, except I enjoyed the album a great deal and went back a second and paid $5 for it.

I don't understand digital downloads. They are an incredibly bad deal. Buying an entire album on itunes makes no sense at all. it costs nearly as much (sometimes more for an older album) as a CD except you don't get an actual physical albums, sounds far inferior because of the low bite rate they come in, almost always has DRM on it and if you loose it, you can only redownload the songs once.

The fact that they are successful makes me think people who care about sound quality are few and unimportant to the recording industry.