Digital Noise: Music and Tech

When Microsoft hired Qi Lu to run its online business last week, the company trumpeted the fact that Lu holds 20 patents.

Patents are far from rare at Microsoft--many developers and researchers hold them--but the online business has typically been led by people with a business or marketing background. That hasn't been working out too well, so it's putting a geek in charge.

Qi Lu

(Credit: Yahoo)

The Seattle Post Intelligencer's Microsoft reporter, Joe Tartakoff, did a little digging on Tuesday to uncover exactly what kinds of patents Lu holds. Most interesting to me, one of them relates to music.

Specifically, it describes a PC application that could take a snippet of a song or audio file, break it down into component parts, analyze them, and then recommend similar songs.

It sounds superficially similar to what Shazam does, but the method is very different and more complicated. From what I can tell, Shazam simply takes a sound sample and matches it against a database with millions of audio files. Getting a fast result requires some fast data crunching, but there's not much deep analysis going on there.

Lu's patent (shared with two other engineers) proposed breaking the song all the way down to very small components like measures and individual notes, analyzing those components to find patterns--for example, a repeated sequence of notes might be the refrain or chorus--and then analyzing the relationships among those parts.

For instance, a pop song is typically constructed of several repeated verses and choruses, with a bridge somewhere in the middle. This is how the application would be able to identify and recommend songs that are similar to the song being played.

Instead of Shazam, the end result might have been more like Apple's recently introduced Genius feature, which builds playlists of songs based on the song you're currently playing.

I suspect that Apple's relying on data from all its iTunes users (Genius asks to collect data about your playing habits) and song meta data--for example, it often recommends songs by the same artist, or other artists in the same genre, or other songs released in the same era. That's much easier--both to program and for your CPU--than trying to analyze audio data for patterns.

Lu received this patent in 2000, which means that he was probably working on it several years before that. Check it out.

Earlier today, a colleague chided me for not highlighting some potential privacy issues in my post about Genius, the music recommendation feature in the latest version of Apple's iTunes application.

First, he didn't like the idea that he had to agree to send Apple information about his iTunes library from time to time. But as I briefly noted, this is how Genius works--it looks at the tens of millions of iTunes users out there to figure out who's playing what when, and matches up users with similar collections and playing habits. Apple claims that this information is anonymized, but even if it weren't, what's the big deal? There's nothing in my music collection or playing habits that's particularly sensitive. OK, maybe you don't need to know that I'm a secret fan of Miami Sound Machine's "Conga," but that knowledge won't let you clean out my bank account or deny me health insurance.

My colleague also didn't like the fact that Apple asked him to sign up for an iTunes account, and in that process asked him for his name (which he might have faked), e-mail address (that's what spamcatcher accounts are for), and a valid credit card number (aha!). I've had an iTunes account for so long that I'd forgotten how the sign-up process went, but going through Apple's iTunes Credit Card FAQ, I checked my credit card info and found that I hadn't entered any. That's because I used a gift card code to sign up, an option that the FAQ says still exists today.

I know why Apple requires an iTunes account for Genius: if Genius recommends a song from the iTunes Store, Apple doesn't want to interrupt your potential impulse buy by making you enter a credit card number at that time. It's kind of like a music subscription service...only with a per-download charge model. (Don't get carried away.)

But the iTunes requirement and sign-up screens made my colleague feel like Apple was asking too much for what he was getting in return. Fair enough--I usually enter fake names and e-mails for newspaper sites that require free registration, and I don't give any information at all to certain Web sites (like Russian MP3 download sites). We all have different comfort levels.

I was actually a lot more concerned when the clerk at the Apple Store asked me for my Social Security number when I was buying an iPhone. This is standard practice for all cell phone providers--they're essentially forwarding you a lot of value (a phone subsidy and a month of service before your first payment), so they need to run a credit check. What bothers me is the fact that a government ID number has become a universal identification number for all sorts of private-sector entities that have no relationship with my tax bill--insurance companies, landlords, cell phone providers, the works. Having all that data available with one number is a boon for identity thieves.

But, you know, I wanted that iPhone. The clerk let me enter it on the screen myself while he looked away, so he couldn't have memorized it and used it to steal my identity. But he mentioned that he'd lost two sales to people slightly more paranoid than me when the SSN requirement came up.

Getting back to Genius: I still like it. A lot. But apparently the latest iTunes update, 8.0.1, is causing some users to have iPhone syncing problems, so proceed with caution.

Although I was expecting more from Apple's latest announcements, I've had a few days to play with the Genius feature introduced in iTunes 8 and I'm quite impressed. Pick any song, and it builds a playlist of other songs that fit well with that one. It's a fun and useful feature for those of us too lazy to build playlists by hand, and bored with the auto-playlists (such as by genre, song length, or date added) and the random shuffle feature.

When I picked this Clash song from London Calling, Genius didn't just limit its suggestion to other punk and post-punk bands, but threw in the Stones and Dylan too.

I was worried that the feature would be too obvious--for example, that it would just recommend songs in the same genre or by related artists. In fact, it's much more interesting than that. I'd guess that it's comparing data from millions of other iTunes users who have signed up for the service (part of the install process sends information about your library to Apple), then matching songs that appear frequently with the selected song. It's probably also using my own playback data, favoring songs I've played back to back and have not skipped.

For example, when I picked a Love and Rockets song, it came up with some predictable picks--two more L&R songs, Siouxsie and the Banshees, and Catherine Wheel--but also had some outside picks like Ministry, The Pixies, and Sisters of Mercy. All of these songs might be broadly categorized as "music that loud-rock fans in their late 30s liked in their wasted youth," but I wouldn't have put them together.

Other experiments revealed a similar mix of expected responses and surprises: a search on a George Harrison tune from All Things Must Pass not only threw up other classic rockers Neil Young and Steely Dan, but also modern-country chanteuse Neko Case and ethereal country-psych outfit My Morning Jacket. Very nice.

I did have some problems getting the Genius feature to recognize certain artists, like Fantomas and Fela Kuti, and my attempt to update the Genius feature from the Store menu resulted in an error message, as it couldn't find some mysterious folder it was looking for. (On my hard drive? At Apple? Who could tell?)

There's also a Genius Bar that recommends similar songs from the iTunes Store that aren't in your collection already, but I hardly buy any music online, and many of the recommendations are already in my LP collection but not yet digitized. So this feature doesn't do much for me, but could be useful for folks who buy lots of music online.

At any rate, it's a free update to a free piece of software, and I can recommend it without reservation for Windows XP users. Vista users, however, might experience the dreaded Windows blue screen o' death when they upgrade and connect their iPhone--check out iPhone Atlas for some possible fixes.

I'm following Tuesday morning's Apple event, and while HD TV and NBC's return to iTunes are nice, this blog's about music. So here are some instant reactions to the music-related news:

Genius playlists. Select a song, click this feature, and you'll get a playlist from your collection of similar songs, plus iTunes recommendations. This sounds like a promising way to solve the fatigue that a lot of users develop with their own collections after a few months, as well as the jarring transitions that sometimes pop up during a random shuffle session. Of course, so much depends on how well it works--will the algorithm adhere toward the same artists and genres? Will it be reliant on metadata such as composer and year, or can it analyze actual audio characteristics like tempo and key? The latter would be amazingly cool. I'll check it out later Tuesday and post more details then.

Bright colors and a nice oval shape characterize the new line of iPod Nanos.

(Credit: James Martin/CNET Networks)

New iPods. The super-thin Nano looks amazingly cool and will probably do well this holiday. And a new 120GB Classic for $249 matches what Microsoft announced for the Zune on Monday. Only it's an iPod, which means it looks way cooler, has a scroll wheel, is far more fashionable...you know the drill. They've put the cover flow and accelerometer from the iPhone in, meaning the Nano will know when you turn it on its side. You can shake it up and it'll shuffle songs. New bright colors, and more recyclable than ever before. (Wow--is it smart to introduce a new product by talking about how you'll have to throw it away someday?) This is all fine and dandy if you don't have an MP3 player already, but I'm not seeing any exciting new features that will make existing iPod owners run out and upgrade.

The iPod Touch is also getting a makeover, super-thin with an integrated speaker, has the App Store (games!) and Genius and Nike+ features built-in. This is really becoming a hybrid entertainment device--more like a PSP than a mere MP3 player. Still, I've always thought this product fits into a weird niche between the iPhone and old-fashioned iPods or other MP3 players. I guess it's for people who want cutting-edge technology (touch screen!) but aren't ready to part with their current phone. BlackBerry fans, maybe. They seem to be targeting it at gamers, but with an entry price of $229 I'm not sure that will work--that's $100 more than the cheapest Nintendo DS Lite, about $60 more than the cheapest Sony PSP, and--heck--$20 more than an Xbox 360 Arcade. Then again, it is the market-leading MP3 player as well as a game device.

That's it? OK, that's it.

Harrumph. I'm happy to hammer Microsoft when it screws up, as it did with some earlier Zunes, but I have to say that this week's Zune announcement was a lot more exciting than the iPod announcement. The hardware's nowhere near as cool--Apple's a great hardware and design company--but I'll be darned if the new Zunes don't do more when it comes to music. They're not all-in-one devices like the Touch is becoming, and Microsoft certainly doesn't have anything that can touch the iPhone, but on a pure MP3 player basis, the Zune's finally looking competitive.

That said, Apple's probably done enough with the new designs to keep the iPod Nano flying off store shelves this holiday. And Zune's got a bad reputation to overcome.