August 9, 2006 4:00 AM PDT
Researchers take the blur out of shaky photos
The technique is based on an algorithm that deduces the path that a wobbling camera took when a photo was shot, then uses that path to reverse much of the resultant blurring. The method isn't a miracle cure, but researchers at the Massachusetts Institute of Technology and the University of Toronto have used it to significantly help a wide variety of sample images.
One example shows a bird with black, white and rust-colored feathers, blurred to the point where its legs are barely discernable. After processing, it's possible to see not just the legs, but also a dark patch around its eyes, white patterning amid the black feathers and other details.
"This is the first time that the natural image statistics have been used successfully in deblurring images," lead researcher Rob Fergus of MIT said in an interview after a demonstration at the Siggraph computer graphics convention last week. The authors have filed for a patent on the process.
The technique, which takes 10 to 15 minutes for typical images, uses a statistical property that describes transitions from light to dark in the photo, Fergus said. That property is the same for all real-world images, so by seeing how it varies in a particular photo, the process can infer the camera motion.
Image processing is a big business, and compensating for human and camera error is a significant part of this. Image-editing software products, and even some cameras, can routinely remove red-eye problems caused by flash photography. A big selling point in new cameras is technology to counteract the unsteady hands of photographers as the photo is being taken. And numerous plug-in modules exist to help Adobe Systems' Photoshop with photo improvement tasks such as removing the speckles of image noise, or sharpening edges to make images crisper.
Right now, Photoshop's latest version, CS2, comes with some deblurring technology in its "smart sharpen" filter. It can compensate in a limited way for focusing problems and for image blur, if the camera was moved in a straight line.
The researchers' approach, in contrast, deals with more complicated jiggling motions. "The real patterns really are weird," Fergus said.
Dave Story, vice president of digital imaging product development at Adobe, believes the group's work is a step in the right direction. It's a little nicer that what we've seen before," he said. "You can start to more accurately and more automatically judge which way the blur was coming from, and it can handle nonlinear paths."
However, the process still leaves some artifacts in the image, meaning more work is necessary, Story said. "We've been exploring this area for three or four years, and there continue to be challenges in making this predictable enough that people will want to use it and it doesn't produce any unnatural artifacts," he said.
For example, when testers are shown sample images, they say that people look "creepy and kind of unnatural," he added.
A study in contrasts
Fergus' technique takes advantage of a statistical property of snapshots that remains constant even across widely varying photos. The property is the collective measurement of the differences in brightness from each pixel to its neighbor.
"It turns out that images in the real world tend to have a distinctive distribution (of light-to-dark gradients)," Fergus said. "If you take lots of different photos, the distribution (of gradients) is very similar--it doesn't change a huge amount between what you would think are different images."
But random images generated by a computer, for example, have a different distribution, Fergus said. "Real images just aren't like random points in million-dimensional space. They have a certain structure," he said.
2 commentsJoin the conversation! Add your comment