Crave

JPEG XR, an image format created by Microsoft that promises a number of advantages over JPEG, has cleared a key standardization hurdle.

The Joint Photographic Experts Group, which standardized the original and still ubiquitous JPEG format, sent JPEG XR to the "final phases of standardization" after a vote at a January meeting, the group said Thursday. That means the standard's future is more certain.

"The committee expects the JPEG XR International Standard to be published later this year," the group said.

JPEG XR offers a few advantages over JPEG, according to Microsoft. For one thing, as the XR "extended range" abbreviation suggests, it offers greater dynamic range--the span between the brightest brights and darkest darks in a photo.

JPEG uses 8-bit encoding that provides 256 gradations, but JPEG XR can use 16 bits or more for finer distinctions and more editing flexibility. Newer digital SLRs typically record 14 bits data, and the hobbyist practice of combining multiple shots into a single high-dynamic range image also benefits from more bit depth.

Another advantage of JPEG XR is that it uses a more efficient compression algorithm that provides either twice the image quality as JPEG at the same file size, or half the file size for the same quality, according to Microsoft. And unlike JPEG, setting JPEG XR to record at its highest quality level loses no information to compression artifacts.

Last, it's easy with JPEG XR to decode just a portion of an image, making it faster to zoom in on an image, and Microsoft designed the technology to work well baked into camera image processors' circuitry, not just to run in software.

Microsoft hopes JPEG XR will become widely used, but it faces a huge challenge in displacing conventional JPEG. It's taken the first steps, though: Windows Vista supports the format on which JPEG XR is based, called Windows Media Photo and later HD Photo. Microsoft also has released HD Photos support for Photoshop and Mac OS X

(Via Bill Crow)

It's a boon that digital photos can incorporate textual information, leaving behind some film-era complications, such as having to separately record a photo's caption or copyright status.

But there are some problems handling this so-called metadata, and now Canon, Adobe Systems, Apple, Microsoft, Sony, and Nokia have banded together to solve some of them.

The companies have formed the Metatdata Working Group and released a first set of guidelines that attempts to standardize some issues that can crop up as metadata travels from cameras to computers, software, and Web sites. On Wednesday, the group announced its work at the Photokina camera show in Germany.

"Whether you're a soccer mom uploading photos to a Kodak gallery, or a pro selling images on Getty, these are issues everybody deals with," said Josh Weisberg, Microsoft's director of digital imaging evangelism and the metadata group's chairman and founder.

For example, when moving a photo from one application to another, a vertically orientated photo can get rotated 90 degrees into a landscape orientation, or captions and descriptive keywords can get lost. Part of the problem is that there are multiple ways to record metadata, including EXIF (Exchangeable Image File Format), IPTC (International Press Telecommunications Council), and Adobe's XMP (Extensible Metadata Platform).

The working group has produced guidelines to try to bring common practices to metadata areas including keywords, description, creator, star rating, orientation, and location, Weisberg said. The group dealt with three file formats: TIFF, JPEG, and Adobe Photoshop's PSD.

The Metadata Working Group's guidelines are a free download from the Web site, and anyone is free to implement them without worrying about infringing any of the members' intellectual property, Weisberg said.

Being guidelines, others are free to handle metadata they way they want, but the collective clout of the working group members--the two major operating system makers, the top camera maker, and the top image-editing software maker--mean it's likely others will follow suit.

Up next: Handling raw images
There's more work to be done, though.

The working group got started on the current guidelines a year ago. Now, it's moving on to the next set of issues. "With the first version, we began with consumer scenarios. We're formulating a plan for a second version. It's our intent to address professional scenarios," Weisberg said.

One big issue is handling the profusion of raw file formats produced by higher-end cameras and commonly used by professionals and advanced amateurs. These formats are generally proprietary, so it's hard to handle their metadata. Windows does so by relying on software supplied by camera makers, but Adobe and Apple do their own reverse-engineering work to handle the metadata. So for example, unless a Windows Vista user has downloaded the appropriate support, the operating system's file browser software can't report when a raw photo was taken, even though that metadata is stored in the photo file.

"It is a goal to try to establish guidelines for where and how metadata is stored in raw formats," Weisberg said.

Another possible issue is handling metadata for photo licensing information, which could bring some rights management order to the today's image copying free-for-all, but that's tricky. "We're in the phase of capturing the problem," Weisberg said. "There are no standards in the industry for licensing images that are widely adhered to."

(Credit: Lori Grunin)

When Canon's Chuck Westfall popped by last month to give us a sneak preview of the EOS Rebel XSi, I expressed my disappointment that the camera's raw continuous-shooting maxed out at a mere 6 frames. I asked Chuck if there was anything on the horizon that might deliver a better raw burst-shooting experience for the can't-afford-a-Mark-III set. His immediate and somewhat unexpected response: "JPEG XR." A few weeks later, I put a similar question to Casio's Scott Nelson, a product manager in the camera division, as he showed off the company's burst-mode blitzkrieg, EX-F1--a camera which doesn't support raw for burst shooting. He, too, indicated that JPEG XR held the key for delivering higher-quality, high frame-rate photos at reasonable prices.

Never let it be said that I couldn't recognize a trend once it whacked me on the head a couple of times.

Raw files--data straight from the sensor--place a heavy performance burden on a digital camera. Though they're same dimensions as JPEGs, raw files support 12-bit or deeper color, while JPEG and its widely ignored successor, JPEG 2000, support only 8 bits. That makes the raw file footprint bigger, even when compressed, and increases the required amount of buffer memory. Furthermore, while JPEG-processing chips are cheap, the proprietary nature of raw files makes it necessary to use dedicated silicon for processing them with any speed. That's a lot of cost to add to a sub-$1,000 dSLR or enthusiast shooter.

Microsoft's JPEG XR--the XR stands for "extended range"--provides some of the image-quality benefits of raw while offering the smaller file size and non-proprietary processing benefits of JPEG. Keep in mind that JPEG XR doesn't replace raw. It simply offers better compression algorithms, and a wider dynamic range than JPEG. At best, one might find a JPEG XR photo visually indistinguishable from a processed raw file. That lets it stand up to retouching better--suffer from less degradation--than its predecessor. For shooters in that market segment, that may be enough.

Clearly, no one's talking about actual products yet, and JPEG XR-as-standard hasn't even attained Committee Draft status within the ISO's JPEG committee (that's scheduled for the end of March). But the fact that I'm actually hearing about it in conversation makes me think that we might start to see some implementations by next year. While I still would prefer longer raw bursts, JPEG XR strikes me as a reasonable compromise.

I'm among the legions who fume when the investigator on the TV show zooms in endlessly on a photo to uncover some minute detail that in reality couldn't have been photographed by any camera. Worst is when the investigator clicks some "increase resolution" button to smooth a bunch of blocky pixels into a richly detailed image.

This low-resolution image shows the greater detail that can be shown in the license plate by combining data from several frames of a video. The lower view of the plate is enhanced.

(Credit: MotionDSP)

Although that Hollywood hokum is an information-theory impossibility with a single image, some limits are lifted when you have multiple shots of the same scene. And a start-up called MotionDSP is working on commercializing that technology to improve photo and video quality.

The image above demonstrates the technology in action. Clicking the arrow buttons will load different images; my favorite is the mariachi band in the gazebo, in which the process reveals arches and architectural details otherwise lost in noisy murk.

The technology also can get rid of chunky compression artifacts, smooth jagged lines, enrich colors, reveal details, and make text readable. It's an example of computational photography--or videography in this case--in which sophisticated computer processing can improve a photo or video after it was taken.

MotionDSP has been funded by In-Q-Tel, the Central Intelligence Agency's venture investment arm, which naturally is interested in software to extract information from grainy or low-resolution images. But the San Mateo, Calif.-based company is raising a new round of funding to underwrite a more consumer-oriented application of its software, said Chief Executive and founder Sean Varah.

The company showed off its FixMyMovie.com technology for improving lower-grade video at the Demo conference last fall, but the company has other applications of the technology in mind, Varah said.

Most recently, the company added the ability to create a single high-resolution 1280x1024 JPEG image from a 320x240 video. The company also is contemplating use of the technology to stitch together smaller frames into a wide panorama, to improve image tones to retrieve detail in bright highlights and murky shadows. And it's possible, if there's demand, that the company could pursue resolution-enhancing technology to let photography enthusiasts improve their images, Varah said.

Improve your photos?
MotionDSP's technology works by comparing as many as 25 views of the same subject matter. The FixMyMovie site uses the consecutive frames of a video, but the technology also works on a collection of still images.

A burst of five or six images--"it's better if your camera moves a bit"--can be combined into a single still image with four times the resolution, Varah said.

"If you have good 10-megapixel image, do you need to make something bigger? It might make sense if you want to crop or make a billboard," he said.

I suspect there's a significant population that might be interested; some purchase tools like OnOne Software's Genuine Fractals to increase the pixel count of their photos for large-printing purposes. The MotionDSP method might not be a simple process, though, for example in a case with moving subjects.

For videos, FixMyMovie can make several improvements. A video shot with a cell phone at 7.5 frames per second, for example, can be increased to 15 frames per second.

Right now, the Web site is free, but eventually MotionDSP will move it to a "freemium" model in which customers would pay for improvements to longer or higher-resolution videos, Varah said.

Better video from set-top boxes?
MotionDSP also is exploring licensing deals that could enable companies to embed the technology in devices such as set-top boxes. "Everyone wants to take Internet video to the television," but today's low-resolution YouTube videos aren't inspiring on a large screen, he said.

Right now the software takes a little time to improve videos, but with multicore machines growing more common, on-the-fly processing will arrive soon. "I think real-time is less than a year away," Varah said.

The company also is seeking new investors. "We're out raising a round now to take FixMyMovie and really expand on it," Varah said.

One area of interest is building an online service that can be embedded elsewhere--Facebook, for example. Another is improvements to the FixMyMovie site that would let users automatically push videos to one's YouTube account or a blog.

MotionDSP got started about three years ago with technology from the University of California-Santa Cruz. The company now has 18 employees, with much engineering work done in Serbia.

Update: I clarified the caption of the illustration to better indicate what editing had gone on to produce the side-by-side images.

Microsoft has taken the beta tag off a plug-in to let Photoshop read and write files in the HD Photo format, which Microsoft is standardizing as JPEG XR.

The free plug-in is available for download for Windows and Mac OS X systems. The plug-ins work on Windows XP and Vista, Mac OS X 10.4 and 10.5, and Photoshop CS2 or CS3, Bill Crow, who's overseen the HD Photo and JPEG XR effort, wrote on his blog Thursday.

These two images both are edited versions of overexposed originals. After editing, the overexposed JPEG version on the left looks murky. The right picture, originally encoded and then edited as an HD Photo, has more dynamic range, so detail in the highlights can be recovered better. It's shown here converted back into regular JPEG after the editing process.

(Credit: Microsoft/Bill Crow)

Microsoft hopes HD Photo eventually will replace the ubiquitous JPEG standard overseen by the Joint Photographic Experts Group. Among the HD Photo advantages that Microsoft touts: it offers more efficient compression, richer color and a much wider dynamic range; it can optionally store images without data loss from compression; it's free of royalty and licensing constraints; and it can run in camera hardware. Support for the file format, initially called Windows Media Photo, is built into Windows Vista.

HD Photo also can be used to show images online at different resolutions, transmitting only the portion of the image that's shown on the screen. That's useful for zooming in to a high-resolution photo without having to download a vast image, a technology Microsoft uses in its HD View software for viewing detailed images online. One organization using HD View is Xrez.

However, Microsoft faces significant challenges in encouraging adoption of the technology. Building it into Vista is a big step, and an endorsement from Photoshop publisher Adobe Systems helps, but JPEG is deeply entrenched. Standardization through JPEG could encourage industry players to adopt the standard--in particular those who are leery of Microsoft's power.

But there are plenty of standards that never catch on. What could really tip the balance in favor of HD Photo/JPEG XR is if it gets built into cameras directly so photographers can start using it from the outset.

The final version of the plug-in, developed in part by Pegasus Imaging Systems, looks mostly like recent betas, Crow said.

"All the changes we've made since the last beta are under the covers, fixing a couple minor bugs, addressing several theoretical security vulnerabilities and generally bringing the code up to current Microsoft standards for released software," he said. "Don't forget that the beta versions will expire on December 31st, so you should definitely download and install these new released versions."

A new attempt to provide a higher-end sequel to the ubiquitous JPEG image standard is officially under way.

The multiple countries participating in the Joint Photographic Experts Group, which created the JPEG standard, have approved an effort to make Microsoft's HD Photo format a standard called JPEG XR, said Bill Crow, who has led Microsoft's HD Photo effort and who just took over the company's Microsoft Live Labs Seadragon imaging project. XR stands for "extended range," a reference to the format's ability to show a wider and finer range of tonal gradations and a richer color palette.

"The country vote is done, and it passed," Crow said. "That means the International JPEG committee has decided to go ahead and create the standard. Now it's just a process of doing that work," a process that will begin later this month in a meeting in Kobe, Japan.

The move is an important step in the transformation of the photo format from an in-house technology called Windows Media Photo to a neutral format more likely to be palatable to companies that don't want to be beholden to Microsoft.

However, the move also means that Microsoft will have to be more patient with its hopes to get HD Photo to catch on more broadly. Standardization "typically takes around a year," Crow said.

The wait is worth it, said Josh Weisberg, director of digital imaging evangelism for Microsoft's Rich Media Group.

"As much as I would love to have more support for it, I think it's logical for people to wait for there to be a standardized version of it," he said. "If we weren't going through the standardization process, we'd be pushing much harder for people to support it."

In Microsoft's view, HD Photo also offers better compression and support for in-camera image processing. It's built into Windows Vista, but Microsoft offers the software development kit to implement the technology free and with no royalty constraints. Image-editing powerhouse Adobe Systems has voiced support for the format.

Microsoft announced some significant progress Tuesday in getting its HD Photo technology standardized as JPEG XR, a significant development for photographers like me who don't like the idea that their camera is discarding data when it converts image sensor information into a JPEG.

But the arrival of a higher-quality alternative to conventional JPEG could mean a bit of a turf war between Microsoft and Adobe Systems, which is trying to popularize a file format called Digital Negative (DNG). DNG is, in part, an attempt to bring some order to the chaos of proprietary "raw" image formats that higher-end cameras produce, giving photographers access to sensor data that hasn't been boiled down into a JPEG.

(Credit: Adobe)

Microsoft positions JPEG XR chiefly as a higher-end replacement for JPEG, but in talking to Robert Rossi, Microsoft's principal program manager for emerging image and video technology, his opinions about JPEG XR's relation to raw and DNG jumped out at me.

Adobe's vision for DNG is that increasingly sophisticated software will take the hassle out of processing raw images, enabling DNG technology to spread more broadly. But Microsoft seems to believe JPEG XR will handle the needs of enthusiasts demanding more quality.

With JPEG XR, "You're giving people much of the capability of raw in a convenient file format. On the ultra-high-end there might be still a preference to use raw," Rossi said.

DNG has "a far more limited market or focus," Rossi added. "We are kind of approaching the raw/DNG functionality, but we would go much lower into the prosumer and consumer market, all the way down to cell phones."

Ed Lee, an analyst with InfoTrends, sees some competition between DNG and JPEG XR. "I think some of it comes around to who does the better job marketing the format and getting it adopted," he said.

Personally, I'd welcome a little competition among powerful companies trying to improve image quality, as long as the world isn't saddled with two competing standards that do the same thing. But although there's definitely some overlap, I suspect the two formats will remain more in separate domains--and not just because Adobe spoke positively about JPEG XR earlier this year, indicating it doesn't feel too threatened by JPEG XR.

If it fills Microsoft's expectations, JPEG XR will be used in a much larger, mainstream photography market. DNG and raw, in comparison, appeal chiefly to professionals and advanced amateurs today, and no matter how easy processing those images may become once downloaded from a camera, any amount of processing will rule out a large population.

JPEG XR does address one advantage of raw and DNG, the ability to preserve more of the original data from image sensors. JPEG retains 8 bits of data for the blue, red and green in each pixel, but cameras typically record 12 bits, with Canon's new 1D Mark III recording 14 bits and higher-end models 16 bits.

JPEG XR, though, has immense bit depth--with 16 or 32 bits of data recorded for each pixel's color, that means somewhere between 65,536 and 4,294,967,296 shades of tonal variation between black and white. Thus the "XR," or extended range, moniker. Regular 8-bit JPEG has 256 shades, which is plenty if they happen to be distributed perfectly, but not enough if you want to use photo-editing software to brighten up a face that's lost in shadow.

However, DNG and raw formats offer something JPEG XR can't: unprocessed data. Creating a JPEG XR image means the camera is making its best guess about color balance--compensating for the bluish hue of fluorescent lights or the orange cast of incandescent, for example--as well as reducing noise and sharpening edges. For those who want that level of control, stick with raw or DNG.