Facial recognition face-off: Three tools compared
Last week's Picasa software update from Google brought with it a neat trick--facial recognition. But it wasn't the first free consumer photo-editing software to find faces. In January, Apple unveiled the latest version of iLife, which included an updated version of iPhoto that could detect and recognize faces in your photos. And this time last year, Microsoft released an updated version of its Windows Live Photo Gallery desktop software that could find faces inside of photos, though it couldn't (and still can't) recognize who's in them.
So, how do these three stack up? To figure that out, we put them to the test. Using 500 sample photos on fresh installs of each program, we tracked around how long each of the tools took to process all the photos, as well as some notable hits and misses from each.
To be fair, our results may not scale, or match the experience you will have. For one, we're using a test bed of photos that's almost entirely 12-megapixel JPEG files, whereas some people may be shooting smaller or larger files that may be in different formats and contain large groups of people--something that can slow these programs down. You're also likely to have a whole lot more than 500 photos sitting around on your computer; we certainly do.
Note: Adobe's PhotoShop Elements software (for Windows | Mac), which also includes a facial recognition feature was not included in this roundup since it's a paid application. Technically iPhoto is as well, but we included it since it comes free on all Macs.
The apps and workflows
iPhoto
iPhoto is the only product of the bunch that's Mac-only. It comes bundled with all new Macs, but the latest version (which includes face detection) must be purchased as a software upgrade if you've got iPhoto '08 or lower. We've included it in this roundup as a free product since it comes bundled with all new Macs.
Face scanning in iPhoto happens automatically, but it's largely a manual process, requiring users to "train" the system to recognize certain faces. The program took around nine minutes to scan through our 500 test photos and when it was done it didn't offer up any suggestions of photos with faces in them.
Instead, users are required to click on a photo with a face in it and hope the program picked it up. If it has, users can simply type the name in--which will auto complete if the person is in your Mac address book. If someone's face was not found, but you can see it in the photo, you can manually contain the face inside of a box, then tag it with their name.
iPhoto's system offers up suggestions of faces it thinks belong to certain people.
(Credit: CNET)After you add names to just few photos, iPhoto's system begins to piece together others that look the same--although it doesn't learn as fast as it does for photos where it already found the faces. In my testing, it only took two photos to get it to offer up some more suggestions. If those suggestions are correct, continuing to add them was just a matter of a few clicks.
iPhoto's system for doing this isn't perfect though. As far as workflows go, iPhoto's requires a fair number of steps. As it offers up suggestions, you need to click an additional button to get into the "confirm name" mode. There's also not a way to skip directly to its suggestions without first scrolling down past the photos you've already confirmed as that person.
Its one grace is that like Picasa, it's fairly easy to bulk accept or reject iPhoto's suggestions--but doing so isn't made immediately evident from the software. The normal system requires one click to accept the person as a match, whereas two clicks reject it. This works great for a handful of photos, but if you're going through hundreds of shots it's a pain. Apple's solution is to have you drag your mouse around the photos you want to accept, while holding the option key while you do it sets all of the photos to be rejected. This makes it easy to blow through a handful of photos at once.
Picasa 3.5
Google's Picasa 3.5 is the first software version of Picasa to feature facial recognition and sorting. It's also one of the easiest programs to use out of the three we tested.
Picasa begins to scan for photos as soon as you fire it up for the first time, as well as any time new photos are added. For us, it took 14 minutes once it started scanning to when it finished, however having recently tested this on a larger library, those times don't necessarily scale. Some users may have to leave the program running for a few days for it to finish scanning.
Once it's done, it presents you with an array of faces that can be claimed by name. These names can be created on the fly, or if you're signed into your Google account--pulled from your Google Address Book.
Picasa offers up suggestions of faces it thinks may be that person, along with notifications on when it finds new matches (seen as the orange question marks on the left).
(Credit: CNET)Now here's where Picasa offers one of the best user workflows out of all of these services. Once you've identified a few photos as the same person, it begins offering up suggestions of other photos where they may have appeared. And instead of making you click through one at a time, it has speedy ways to bulk accept or deny its suggestions.
It does this two ways: one is to give you a little alert with an orange question mark next to their name. It also lets you skip directly to those recommendations while ignoring the items that you've already confirmed as that person--something iPhoto does not. It also lets you bulk confirm all of them at once, which can be a huge time saver instead of clicking the little yes and no buttons that sit below each suggested photo.
Windows Live Photo Gallery
Windows Live Photo Gallery is Microsoft's free Web-enabled desktop photo library management, editing, and sharing software. It's largely the same as the Windows Photo Gallery software that ships with Windows but adds a few extra features like a photo syncing between computers, a panoramic stitching tool, batch resizing tools, and facial detection.
Being the oldest software with facial detection in this bunch did not mean Windows Live Photo Gallery was the slowest. In fact, in our tests, the software came out the fastest, scanning through our test photos in just under eight minutes.
That was unfortunately the fastest part of the process though, since "detection" is about all it does. As for recognizing who is in those photos and putting them into groups, the legwork gets put entirely on the user. This wouldn't be so bad, except for the fact that built-in tools for organizing require that you go through each photo one at a time to name each face it's found.
You can click on multiple photos with the same person and drag them over to your source list of "people tags," but this does not automatically assign whatever face the program found to that person. This leaves you with an additional tag the system has not yet assigned.
Windows Live Photo Gallery is Microsoft's free Web-enabled photo editing and organization software. It can detect faces, although isn't able to recognize the same person across multiple photos.
(Credit: CNET)As for manually assigning people tags to faces it's found, if you're signed into your Windows Live ID it can pull in names from your online address book. This speeds up the process (assuming you have the people in there already), but here again, the user workflow has not been well thought out.
Instead of surfacing recently-used contacts to the top when beginning to type a name, it consistently puts things in alphabetical order. If you have a large address book, or friends with slightly similar names this really stinks. It's also annoying if you're going through a "roll" of photos from an event where you have many of the same people.
One thing Windows Live Photo Gallery does well is offer users a way to categorize their friends into folders. Actually, this needs to be done from somewhere like Hotmail, or another service that lets you access or edit your address book. But the results are great. You can very quickly sort out friends and family from business contacts and coworkers. In Picasa and iPhoto you're stuck with an alphabetical source list.
Winner: Picasa
If we were to choose one of these apps to help churn through an enormous library, Picasa is the way to go. It has, hands-down, the best workflow for finding faces, as well as alerting users to when it had found people in new photos. It's also the only program out of the three we tested that's cross platform, meaning you can use it on either a PC or Mac and have an identical face recognizing experience.
Face detection hits and misses
Hits
Something surprising that just kept happening with all three pieces of software was it picked up small faces in photo frames and posters, as well as people behind reflective glass surfaces.
All three software apps were able to find faces in the background of this photo of CNET News Senior Writer Erica Ogg.
(Credit: CNET / Josh Lowensohn)This happened on a number of occasions but was most apparent when using Picasa since it shows you all of the possible faces it picked up inside of one menu. Many times these were people far off in the background, or in spots so dark that the photo had to be brightened up a bit for us to see that there was a person there.
Also impressive was that all three pieces of software did not pull up photos of the pets that were included in some of the photos from our batch of test shots, as well as from an auxillary test done with 20 photos of the same cat. Blog MacLife had a different experience--at least for iPhoto, and was able to get the software to discern between two cats.
Misses
It should be noted that all three of these programs (and facial recognition in general) has an incredibly hard time with sunglasses, hats and hooded sweatshirts, and to a lesser degree, things like regular glasses and facial hair.
Just like the Unabomber, hiding half of your face with large sunglasses and covering up your hair with a hood is a surefire way to have all of these apps be unable to identify you. That was certainly the case for some of our test photos. We threw in more than 50 shots that included large sunglasses--none of which were picked up as recognizable.
No Windows Live Photo Gallery, this is not a face.
(Credit: CNET)Also problematic were features that were face-like that got picked up by the software, as well as facial expressions. Wide smiles, open mouths, and upturned eyebrows threw off all three pieces of software. Again, this is one of those areas where the software's workflow can make these misses less bothersome.
What was more of a problem though, were the programs not being able to pick up people's faces if turned sideways--be it a profile shot, or someone with a tilted head. For posed snapshots this isn't a big deal, but a good number of our photos featured people who weren't looking at the camera. Out of three programs, Picasa handled that the best, simply highlighting them as faces, even if it was unable to tell whose they were.
Other things to look out for include slightly obscured faces, like people in mid-bite at a meal, as well as faces that are slightly out of focus. This isn't a big problem with point-and-shoot cameras, but if you have an SLR and are using a lens with a shallow depth of field, someone just slightly out of focus can render their face undetectable to the software.
Conclusions
These three pieces of software each offer different approaches to photo management and organization, and face detection is just a small part of the many things they do. It's also hard to truly stack them together, since two of them can only be run only on one kind of operating system (iPhoto for Mac and Windows Live Photo Gallery for Windows).
That said, what matters in the end is the workflow--or how you're able to go through the suggestions it comes up with and use it to help organize and categorize your photos. For that Picasa is the clear winner.
Out of all three products Picasa required the least amount of time and effort for us to properly go through our photos and feel confident that we had tagged them all. It also had one of the best systems for accessing and editing information about stored online contacts by basically sticking the Google contact manager into the program as its own window.
In second place is Apple's iPhoto, which can also make tagging large batches of photos relatively easy, but in the end it lost out to Picasa due to the training required to even begin the tagging process. It also got taken down a notch on the user friendliness scale for not providing more notice when it had come up with more possible face matches.
And in last place was Windows Live Photo Gallery, which to be fair, is limited to face detection and not recognition. My hope is that future versions will take a page from Google's efforts and focus less on the feature itself as much as what users need to do in order to use it for organizational purposes. As it stands the tagging process is just too manual, and without the quick tagging shortcuts and optimizations like Picasa and iPhoto have, users just get muddled down with too many button presses to make it worth their while.
Has your experience differed from ours? Let us know in the comments.
Note: The machine used for testing was a late-2008 Apple MacBook with a 2.4GHz Intel processor and 4GB of RAM. For testing on Windows, the same machine was used but in Boot Camp running an RTM build of Windows 7 Ultimate.
Correction: This post initially misstated the lack of a feature to bulk accept or reject facial recognition matches in iPhoto. The software does contain the feature.
Josh Lowensohn writes for Webware.com, CNET's blog about Web applications and services. E-mail Josh, or follow him on Twitter at http://www.twitter.com/Josh. 
Things like this should be shown at the bottom of the screen or pop up the first first time you go in and then not appear ago until you ask for help and then IT SHOULD be in help.
But I just upgraded to the new iPhoto, and so far, remain unimpressed. What magic incantation to use to get it to start recognizing, or to tell you what it has done?
I have 3 windows systems at my house:
iMac dual boot
dell stock install
self-built workhorse
I was surprised (at least with Snow Leopard boot camp) that windows and apps are a LOT less flaky than either of the other two systems. Windows is really designed to run on an "optimal" setup . . . ironically, Mac hardware is really well suited to make Windows (at least XP) run well. What I was pleasantly surprised at is that since Apple is the hardware vendor, ALL of your drivers are on one disk and ALL of the drivers are properly recognized by Windows. No more popping the box open to figure out what chip is in the computer then guessing which driver on the resource cd (DELL, I'm looking at you here) when you have to reinstall.
I'm not sure if Alienware is as high quality these days as it was 8 years ago, but you've basically got the same type of "matched" components in a Windows Mac which make the experience all that much better.
I'll buy that for a dollar. BTW thanks for the article I've been looking for a nice photo management program and sadly I have to say piccasa was not on my list to try till today.
I simply made a new smart photo album that uses the parameters of only adding photos to that album that have unnamed faces. When I add photos to iPhoto, this smart album will get the photos added to it and then I can just click on that album and see all the faces that have yet to be identified. As I assign a name to those faces, that photo is automatically removed from that album. It works like a charm.
When looking in the "Unnamed" album, it groups faces together, so you can batch accept (comfirm) the entire group. But it is not clear, that it is a group and I have often found errors in the faceses has grouped together.
This made me do alot of errors in my confirmed faces!
Google have to change this!
If you click on "Expand Groups" in Picasa, the unknown faces stop grouping together.
First: I know I can press that button, but I have to remember to do every time I open Picasa.
Second: When pressing the button, Picasa removes the little "add a name" box below each picture and removes the "ignore person" in the upper right corner of the picture.
Third: If I get that many false positives, I would guess that many other get it. It takes time to look though all the pictures you have tagged, to find the once, that Picasa has made you tag the wrong way.
So I stand by my claim, that Google has to change the default behavior, because it makes allot of mistakes. There is a reason, that you have to manually accept all the guesses, that Picasa does. Why don't we have to accept them, when we are tagging unnamed (grouped) faces?
The only facial recognition software i use right now is FastAcess which logs me in with my face. Its not bad.
You know, a lot of PCs can come bundled with Elements, too.
First, in the unnamed people folder, the thumbnails are in constant motion ... This makes it *very* difficult and tedious to get any kind of systematic method of sorting faces going. For example, one approach might be to work through the faces and close ("x" out) the faces that you don't want to catalog. The problem is that as you do this the thumbnails constantly get reloaded and they shuffle around (some sort of unfortunate sort order). This means you are constantly going back to the top and looking all over again. Instead, if the "keepers" could float to the top it would make the sorting of the keepers much more efficient. Just add the new thumbnail groups to the bottom, don't sort them! Better still would be the ability to place or arrange the thumbnail views. This would provide the additional capability of shuffling the unnamed groups around (like those old 15 tile puzzles ;-) so that the same person can be sorted into adjacent cells. From there, it is easy to multi-select and add them to a known person via drag&drop, right click etc.
The other feature that might be interesting is if once you have paired down the unnamed faces you could say "All of these pictures are from people I have already defined, take another shot at mapping them". I would suspect that if Picasa knew that all the unknown pictures had a mapping to already defined people it could change the thresholds on the signature accordingly. The other possible thing to do would be to do a rearrange of the unnamed thumbnails based on similarities. Perhaps an advanced feature would let the user adjust various signature thresholds to result in different possible groupings much like photo editing software allows dragging graphs for contrast and color. Do that superimposed over a generic face to specify which facial regions should receive priority (with real-time highlight of the thumbnails) and you will have a killer UI. The goal here would be to line up similar pictures for easy multi-selection.
Speaking of multi-selection ... Picasa uses the Mac's normal keyboard/mouse behaviors to do multi-select. That is a given. However, several times I lost a large number of selections because I either accidentally held down the ALT key vs the Apple option key *or* I had my mouse positioned on the very edge of a thumbnail and when I did option-click it was read as clicking on the whitespace between pictures. Guess what happens? You lose *all* your selections with no way to get it back. Talk
about frustration ... after spending 10-15 minutes walking through the unnamed photos, multi-selecting and que'ing up a bunch of pics to be categorized, you toss it all. And worse yet, you may not even realize it if you aren't watching your selection preview area in the bottom left.
So, instead of relying on the platforms multi-select, why not have a mode that would say "I am going to start clicking on thumbnails that are all the same person now" and have Picasa maintain the selection independent of standard mouse selection. You might even consider a hold and swipe type of gesture such that selects the thumbnail under the mouse (or deselects if it is selected) The main goal would be to prevent accidental deselection on large multi-screen spanning selections. Once you have your selections you could do a few things, first, you could categorize them directly. Alternately, you might allow the thumbnails to be sorted to the top much like desktops allow icons to be arranged/sorted in various automatic ways.
Oh...and when it crashes or hangs (which has happened frequently), all the naming since starting the program seems to need to be redone???
I did have the same experience with selecting photos and then losing my selection as f1216, until I started using the Hold Selected Items button at the bottom left. Clicking it multiple times as I went through the thumbnails saved me from having to reselect.
Looking forward to loads of bug fixes and a new rev of the product...should be awesome going forward.
Personally, I like Picasa better because I can upload to Picasa web and is free and most importantly, it can run on Windows
I use picasa, too and i'm happy with the software. Happy regards, <a href="http://www.pcmasters.de">Tim Klemt of pcmasters.de</a>
Go away.
Also, why doesn't it show you ALL of the faces, at least the top ten, that it thinks might be the person in the picture. That way I could quickly click on the one that is right?
Picasa seems pretty user friendly...
Many versions ago iPhoto added Keywords that can be manually assigned to a photo.
Added in iPhoto '08 was "Events" and the program automatically splits imported photos into 'events' based upon the timestamp of the photo.
iPhoto '09 added "Faces" as described here
iPhoto '09 also added "Places" that will use any GPS coordinates embedded into into the photo or allow you to manually add geographic location information.
iPhoto's "Smart Albums" then lets you create dynamic groups of photos based upon these various categories.
So I could easily create a Smart Album that contained all the photos with my face, in San Francisco, in the year 2008.
Pretty handy I think.
-Sincerely,
iPhoto Fan Boy
The first scenario sounds better to me as all the work to link the names to the photos will not have to be repeated if you change programs or send the pics to a friend.
Does anyone know the answer to this?
Windows Photo Gallery saves some info into the file - I think. I haven't looked that much at it.
But I would love to have all info written into the photos, so I could share it more easily between programs and friends/family.
- by The Noble Robot September 30, 2009 12:44 PM PDT
- "purchased as a software upgrade if you've got iPhoto '08 or lower. We've included it in this roundup as a free product since it comes bundled with all new Macs."
- Like this Reply to this comment
-
Showing 1 of 2 pages (73 Comments)So my options are: buy the software by itself, or buy a new Mac. This is "free" how exactly?
LOGIC FAIL.