Beyond Binary

Microsoft showed off a technology on Tuesday that could one day allow people to edit artifacts into video as easily as they do with digital photographs today.

The research technology, dubbed Unwrap Mosaic, was shown to do things like adding a mustache and rosy cheeks onto a person in a video. It works by sort of unwrapping a 3D object into a flat image that contains the whole object, in this case a face.

Microsoft demonstrated a video editing technology Tuesday that lets artifacts be edited into video, in this case facial hair, by adding them to a flattened three-dimensional image.

(Credit: Microsoft)

"What we've done is built away of patterning the essence of a video in a single pattern," said Andrew Fitzgibbon, who presented the technology Tuesday at the SIGGRAPH trade show in Los Angeles. The key to that technique is that unwrapped or flattened image."

The effect Microsoft demoed (see video here) is sort of like taking a Woolly Willy to a real person in a moving video. It's applications, of course, could be much broader than adding facial hair (though I have some friends that would probably pay good money just for that particular effect).

While there are plenty of techniques out there for changing colors in a video or other special effects, adding a full mustache, though, is tricky because although it exists in one place--the face--different parts of the face are visible at different times.

Here's the original unwrapped image, pre-facial hair.

(Credit: Microsoft)

In the movies, it's done by using a model of the face. But Fitzgibbon's team was looking to create a single tool that would work on multiple types of 3D objects.

It's still just a research project. Microsoft has released some of the underlying technology into the public domain. Fitzgibbon also hopes to put a user interface on top of the technology and make it available somehow to the public, though he declined to offer a timetable on that.

And this is a still of what the mustached man looks like in the edited video.

(Credit: Microsoft)

Fitzgibbon's paper is just one of several papers Microsoft is presenting at the show. Another, is a follow-up to Microsoft'sPhotoSynth technology, which creates three-dimensional views of an attraction by using lots and lots of still photographs. In the new paper, done by the creators of PhotoSynth, Microsoft talks about how to navigate in a 3D environment constructed from such photos.

For example, one could use the technology to see what the Pantheon looks like from the outside, then zoom in to go through the door, walk down a hall, move up close to see a certain sculpture, turn around, etc. Although this can be done with publicly available photos, say from Flickr, Microsoft said one can also add in personal photos to the mix.