Google gives respite from a raw camera deal
I was in a pinch a few weeks ago, and Google's Picasa software saved my skin. But now my warm glow of gratitude has begun wearing off, replaced by a simmering annoyance with camera makers for their profusion of proprietary raw formats.
Let me explain. I was covering the Photo Marketing Association trade show in Las Vegas, toting my Canon EOS Rebel XT camera to photograph products and people. For my personal photography I usually shoot in raw format to maximize the detail and flexibility, but for work purposes I use JPEG because it's faster to process and CNET News.com graphics are too small to require top resolution.
This screenshot shows a raw image from an Olympus E-3 SLR in Google's Picasa software. At right is the low-resolution JPEG preview, at left the garbled view after an incorrect decoding Google's support for the E-3 is on the way.
(Credit: Stephen Shankland/CNET Networks)But I had a brief moment of panic when I discovered, on a tight deadline, that I'd photographed a Sony full-frame SLR press conference and accompanying photo gallery in raw only. I wasn't happy, because I hadn't installed any software for processing raw images on my laptop. I briefly considered downloading a trial version of Adobe Systems' Photoshop Lightroom, which I use at home, but dreaded the time it would take to get myself to a network connection and install the software.
Then I remembered that Picasa supports some raw formats. Sure enough, it did the trick--after I made my usual end run around Canon, which annoyingly doesn't include a mass storage driver on its cameras, requiring me to retrieve raw files using a separate flash card reader.
Picasa lacked some editing tools I like in Lightroom (and now Apple's Aperture 2.0, too), but I wasn't about to complain.
Until Wednesday.
That's when I received an Olympus E-3 that I'll be testing on an upcoming vacation. The camera has been out since November, but Picasa still doesn't support its raw images.
Raw-support challenges
Picasa showed the low-resolution JPEG preview fine, but as soon as I clicked on the thumbnail, the photo became a speckly mess of pixel gibberish.
For its part, Google said Thursday that E-3 raw support is coming. "We're in the process of testing it and plan to support it soon," the company said in a statement. Picasa uses Dave Coffin's freely available dcraw software, which supports the E-3, but Google said it makes its own modifications "to make it run faster."
It's no surprise Google employs outside software for the complicated task. Olympus told me it leaves programmers on their own to reverse-engineer raw formats: "When asked, we will provide sample raw files to companies, but it is up to them to figure out what to do with them. Our raw format is not difficult, and anyone with any experience with graphic file formats will figure it out in a matter of seconds."
For photographers, there are unpleasant consequences of camera makers' opacity and non-standardization. Programmers from Adobe Systems, Apple, and other companies must toil constantly to support new cameras, and camera makers must develop and support their own software. And the obstreperous nature of raw can curtail the innovation of other programmers, too.
For example, software that can embed location data known as geotags in raw files is much rarer than software that supports JPEGs. Adding metadata such as titles, captions, ratings, and tags is another risky operation; Microsoft Vista can do this, but relies on camera makers to supply software to support their various raw formats.
A programmer's plight
Sachin Garg, a programmer in India, is another example. He's been working on software that can compress raw files more efficiently--about 20 percent to 60 percent more than those already compressed by the camera.
Programmer Sachin Garg
(Credit: Sachin Garg)That's work that conceivably could be useful for those of us with vast archives of raw images, but Garg said the difficulties of working with raw files makes it tough.
"I have started with Nikon's NEF (raw format), and it's a mess. What makes it worse is that even for this single format, there are variations based on each camera, and camera's firmware version," Garg said. "I have managed to read and compress the file, but re-creating the original file again is giving me nightmares."
And that's just one popular format. There are also cameras from Canon, Olympus, Fujifilm, Pentax, Sony, Panasonic, Samsung, Hasselblad, and others to contend with.
"It's a much different ball game to write an algorithm (than just) trying to put it in a practically usable application," Garg said. He understands the camera makers' situation, though. "Looking at each format, one can see the technical reasons why different camera makers are doing things differently and that adopting a common standard can possibly limit the innovations they introduce in newer cameras."
One possible alternative to the raw plight could be HD Photo, which Microsoft is trying to standardize as JPEG XR, a higher-end alternative to conventional JPEG. My guess is that this file format stands a reasonable chance of catching on--especially given the warm response from Adobe and more recently Canon--but even then it's more likely only to intercept photographers just moving beyond JPEG rather than replacing raw.
That's because HD Photo/JPEG XR requires the camera to process the image for de-mosaicking, noise reduction, sharpening, and white balance, all of which are "baked" into the image. For the folks who want total flexibility, they'll stick with raw.
DNG to the rescue?
A more likely alternative is Adobe Digital Negative (DNG) format, a raw format whose specifications are openly shared if not a neutral industry standard. Adobe explicitly created DNG to deal with the raw format "tower of Babel."
But larger camera makers have been reluctant to embrace DNG. It's hard to get firm answers on exactly why not; I'd imagine a variety of factors are involved, ranging from not wanting to be reliant on Adobe or a fixed format to inadequacies of DNG to fully represent raw images. And Pentax, whose SLRs support both DNG and its own PEF raw format, told me that most customers shooting raw use PEF, so users apparently need more convincing, too.
Maybe Adobe just needs to do a little more marketing, standardize DNG, or come up with an improved version 2.0. But for now, the raw format mess shows no signs of being tidied up.
Stephen Shankland writes about a wide range of technology and products, but has a particular focus on browsers and digital photography. He joined CNET News in 1998 and since then also has covered Google, Yahoo, servers, supercomputing, Linux and open-source software, and science. E-mail Stephen, or follow him on Twitter at http://www.twitter.com/stshank. 
It sounds like he doesn't have an original file compressed properly if he can't back out the steps he took to produce the original file.
> file compressed properly if he can't back
> out the steps he took to produce the
> original file.
My first guess was same as what you mention :-), but I have verified that part. Problem is not uncompressing the data, but recreating the original file (while parts should go where?).
Its just that different cameras (even from same manufacturer) do their thing differently, making it harder to track. Nothing too complicated, but just needs some time (actually lots of it, try counting no of camera's released ;-)
<rant style="relation: tangential>
God forbid that they open-source, for example, their lens mounts or flash hotshoe protocols. No, everyone who wants to make something compatible has to reverse-engineer it -- repeatedly. This is less common in the computer business only because people generally won't stand for the most egregious forms of this behavior. Only the most powerful like MS can get away with it.
What's odd to me is that despite all the supposed intense competition in the DSLR space, the trend is getting towards increased proprietariness (a word?) not less.
I for one will cheer the day that someone like Sigma makes *cameras* with Canon and Nikon mounts as well as lenses. Let the big guys put that in their pipes and smoke it.
</rant>
-- dave
It's just arrogance that leads other manufacturers to stick with only their own formats. Nikon even adds encryption, presumably to drive sales of their own software.
File formats and communications protocols (with flashes?) should be common among devices and applications, right? This RAW format stuff is intolerable.
Thanks, Pentax!
software, especially in the image processing area are really pretty
lame. Fuji and, to a lesser extent, Olympus, are way better in
image processing, color and dynamic range issues. Part of the
necessity for shooting RAW in the Canon is all the fiddling you have
to do just to get a decent image. I shoot JPEG in my Fuji S3's and
the extended dynamic range plus decent skin tone rendition have
all but eliminated any need to shoot in RAW at all.
Does your girlfriend work at Google, your best friend? or do you just own GOOG stock?
http://www.news.com/8301-13580_3-9875221-39.html
I don't own any Google stock (unless it's a part of some mutual fund, but I honestly don't know), and my wife and best friends don't work at Google.
I've also written about Microsoft's image viewer (I needed an editor, so WIC wouldn't have helped me much), BreezeBrowser, ACDSee, Phase One Capture One, Adobe Camera Raw, Corel Draw, and probably other raw sofware. I don't own any stock in those companies or have spouses or best friends there either.
Picasa is mentioned as an example of how the tools struggle to keep up with manufacturers hell-bent on chasing their own agenda.
I think it's legitimate to argue for a standard RAW format (although I struggle with how that might stifle innovation if everyone has to subscribe to the same standard). But I think you have easier ways out of your dilemma, and blasting the camera manufacturers for the proliferation of RAW formats ignores your own options at your disposal. They can't meet everyone's needs, nor can they anticipate every situation that a user might find themselves in.
Finding a network was an issue because I was on deadline (there were none at this particular location that weren't encrypted--I did check--and the press room was about five minutes' walk away). For me, at a big trade show right after Sony has announced a full-frame SLR and unveiled it and shortly before I had an interview with a Sony exec, I didn't have 1 or 3 or 10 minutes to spare.
To be clear, I wasn't trying to blame my deadline problem on Canon--I was the one who set my camera wrong. I just think there are lots of problems that ripple out from the profusion of proprietary raw formats, and that particular plight might have been easier in some alternate universe. The personal experience I relayed isn't a "premise" that Canon's raw files left me in the lurch, it's an anecdote to illustrate the plight the industry is in.
understand the technology that you are covering... good to
know... and i'm not referring to shooting raw only, thats a
mistake anyone can make but rather not understanding raw
format.
Windows, Mac and i'm sure Linux (can't say 100% since I haven't
tried my cam on my linux box) all support canon raw format.
But, lets say you had 0 net connection, considering you probably
had to upload your story or email it and with wi-fi spots a dime
a dozen... i'm guessing you do ... but for sake of argument ...
lets say you don't...
I'm pretty sure at photo marketing trade show you could
probably find 1 person who happen to have the canon cd that
came with your camera (or any canon digital slr camera), that
includes both downloading, viewing and editing software as well
as conversion to jpg.
You have shown yourself to be quite inept.
I knew exactly where a network was--in the press room about five minutes' walk away from where I was and I had to return to shortly. Obviously I got to that network to file my story and post my photos, but I didn't have time to waste. I was editing some photos in the middle of Sony's press conference (don't tell Sony), which I didn't want to leave.
I have Windows XP on my work laptop, and it can view raw files, not edit them, and then only if you have downloaded Microsoft's WIC software and installed it, which I haven't. Apple's Mac OS X and Linux were not terribly useful options to me since I had neither installed on my machine.
My point was not that there wasn't software available to me--I mentioned Lightroom, and I could probably have used other trial software, too--rather that I didn't have time to go get it, and Picasa happened to be installed and it only occurred to me then that it could bail me out.
- Compression tool update: working demo now available, finally
- by SachinGargIndia March 18, 2008 3:40 PM PDT
- Just to let you guys know that I have (finally) managed to get the compression tool working on all Nikon and Fuji camera raw files.
- Like this Reply to this comment
-
(19 Comments)20 to 60% compression, totally lossless.
Check it out :-)
http://www.sachingarg.com/compression/sgraw/