Nikon's new SLR leads the pack for sensor quality

The top four SLRs in DxO Labs' current rankings.

(Credit: DxO Labs)

It's not a surprise that the Nikon D3X, the company's brand-new $8,000, 24.5-megapixel SLR, tops DxO Labs' sensor performance test. What is a surprise is the margin by which it leads its rivals from Canon and Sony.

When the French firm unveiled its DxOMark Sensor benchmark test last year, Nikon's D3 was the top scorer at 80.6, a composite number that represents various performance features. Very close on its heels were Nikon's D700 at 80.5, Canon's EOS-1Ds Mark III 80.3, and later Canon's 5D Mark II at 79 and Sony's Alpha A900 at 78.9.

All those cameras were close, but the D3X stands apart with a score of 88. The result shows how much ground Nikon has made up on Canon, which has dominated high-end digital SLR technology.

DxO Labs said cameras must be about 5 points apart for people to see a difference; a 15-point difference is equivalent to a full stop of improvement, meaning that a camera would have the same image quality as a rival with half the available light.

The test measures a sensor's color depth, dynamic range, and low-light performance through a variety of detailed tests of the camera. No lens is mounted during the test of the SLRs, so differences from optics are minimized. Unlike other tests, DxOMark Sensor measures performance based on the cameras' raw-format images, which eliminates changes that the camera or software make when converting the raw data into a recognizable photo.

DxO has real chops in the testing business; the company makes a business selling analytical tools and services to camera makers and others. Don't confuse the DxOMark results with a full evaluation of a camera, though; other significant factors include autofocus, in-camera processing, durability, optics, interface, accessories, customer support, and availability of third-party and used lenses.

The Nikon D3X beat out the D3 and Canon's 1Ds Mark III in image sensor performance. (Click to enlarge.)

(Credit: DxO Labs)

Sony builds the D3X's sensor with Nikon dictating much of the design.

How exactly did the D3 fare better than its top rival, the 21.1-megapixel 1Ds Mark III, which currently goes for about $6,550? A notch better in color depth, which measures the fineness of gradations between different hues, a big notch better in low-light performance, and a very big notch better in dynamic range, which measures how well the camera can accommodate details in bright and dark areas at the same time.

There's been a lot of teeth-gnashing over the price of the D3X, especially given how much it shares with the $4,100 D3, which still beats out its higher-end sibling on low-light performance and shooting speed. But with most technology, top-end products sell in vastly smaller quantities, and the higher up the ladder you go, the more each step costs.

All the top-scoring cameras employ full-frame sensors, which are the size of a frame of 35mm film from days of yore. The attendant larger pixels and extra light-gathering ability allows much lower noise. Most SLRs sold use smaller, vastly cheaper sensors, of which 200 can be carved from an 8-inch silicon wafer compared to 20 many fewer full-frame chips, according to a Canon white paper on sensors (PDF) from August 2006 that's still relevant. (See update below for a discussion about those numbers.)

Compact cameras use yet-smaller sensors, which shows in part why the performance even of high-end models such as Canon's newer PowerShot G10 significantly trails even elderly, low-end SLRs.

The next big shake-up on the DxOMark list should come in the next few weeks, when DxO Labs adds results from medium-format digital cameras, whose sensors have about twice the area of full-frame SLRs. That will help show just how big a competitive threat these high-end, high-megapixel SLRs are for the studio photographers who are the biggest customers of the super-expensive medium-format models.

DxO Labs also added a new reviews section that describes cameras. So far the D3X is the only camera so featured.

Update 2:06 p.m. PST: Brace yourself for a big tangential digression. Illuminata analyst Gordon Haff, in a chat with me, pointed out that the Canon wafer comparisons from the white paper seem off-base, and I agree.

Here's how the white paper expresses the figures comparing the smaller sensor size, called APS-C, with full-frame: "If the sensors are APS-C size, there are about 200 of them on the wafer, depending on layout and the design of the periphery of each sensor... Full-frame sensors? Just 20." Next to those words is an image of an 8-inch wafer.

The area of an 8-inch wafer, also known as a 200mm wafer, is about 31,416 square millimeters, which isn't enough for even 95 APS-C sensors, much less "about 200." Newer chip manufacturing processes use 300mm wafers with an area of about 70,686 square millimeters, and that would be big enough for 200 APS-C sensors, each of which has an area of about 330 square millimeters.

Indeed, when I asked him about it, Canon Technical Adviser Chuck Westfall told me, "The machine that Canon uses to manufacture its CMOS image sensors for EOS digital SLRs does in fact use a 300mm wafer...Canon has had the capability to use 300mm wafers since at least 2006."

Westfall stood by the figures in the white paper, but in my view the comparison is misleading. It compares how many APS-C wafers fit on a 300mm wafer to how many full-frame sensors fit on a 200mm wafer. But certainly the same overall economics apply: more APS-C chips than full-frame will fit on a given size wafer.

[denisevans]

It is pretty clear that if you go through the individual component scores there is something very wrong with the final DXO summary scores. If you look at the individual performances on SNR 18%, Dynamic range, Tonal Range and then colour sensitivity, it is pretty clear that the 5DII and the D3x are pretty similar except that the 5DII allows you to go to higher ISO's. In the main the D3 is about 0.5 - 0.3 stop better than the 5DII at high ISOs big this gap rises to > 1 stop at the lowest ISOs. This 1 stop+ disadvantage for the 5DII and the D3X is largely irrelevant because at these low ISOs 100- 200, you can't actually see noise. You need an instrument to measure it! In summary I have no idea how DXO justify their final summary scores.

[LOOP909]

Is it pretty clear!!??? I think its pretty clear you shoot Canon! Stop Whining!

[willie0343]

It is apparent that this test does have some merits.I had the chance to see the high ISO performance on both Canon's 5D mkll and Nikon's D700 and I could say there is a better IQ in the Nikon's photos in the higher ISO shots, yes we are told to take things with the grain of salt when there is something that you will review ,but the truth is I find that their reviews have been accurate. I am a Canon user but the truth is that for the average eye the Nikon to me has slightly better imaging than Canon.

[inissila]

In the shadows you can see noise - the dynamic range improvement with the D3X at low ISOs is a key feature of importance to landscape photographers who sometimes have to merge several images taken at different exposures to get an acceptable print. The D3X's top dynamic range at base ISO is in my opinion a major feature combined with the high resolution.

[xophaser]

About time cnet do an indepth quality image review, like some time back with all the image comparsions. Lately, they been just stating that it is good or ok image and spending more time reviewing the features.

[denisevans]

For a serious technical discussion of sensor comparisons see:
http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/#model

I think that in all seriousness DXO needs to re-examine its final summaries. The individual test checks seem completely consistent with "clarkvision.com". DXO's reputation is on the line here. This is no small disparity! When you claim you are technical and you publish on the web you'd better not make any major mistakes!

[LOOP909]

Another Canon user!

[LOOP909]

Clarkvision!!??? The site is for retards!

[fafoua]

Hi Denis Evans, I totally disagree with what you say about low ISO performance. It all depends on how you use your camera. I don't do action photography. I use my Pentax K10D almost exclusively at ISO100. It has an amazingly good quality there. With a full frame sensor, I won't go higher than ISO400.
Why ? Because I do a lot of post-processing, bringing back details in the dark parts of my images, where the low ISO noise is present. The lowest the noise is as the base ISO, the best it is for my photography. I don't care about high ISO performance.
BTW, I'm totally confident with DXOMark methodology and accuracy.
Regards, Fafoua.

[dandear99]

Canon is absolutely junk! I've had 4 of their digital DLR's and they've all had image flaws. Worse, they hide it from the public because they are fearful from losing market share to Nikon. They will not let you speak to anyone in their Technical Service Dept.

[ppolgar]

I have been professional photographer for over 25 years. I happen to use both Canon and Nikon. However, I have never seen the level of disrespect and the bashing of Camera brands like I have seen in the last couple of years. In the film days, the emphasis was placed more the skills of the photographer more so then the equipment they choose to use either in the field or in the studio. For someone to say that Canon?s are absolute junk immediately illustrates the individual?s ignorance in the field of photography. Canon has developed several benchmark cameras in the past few years. Personally, I own both the Nikon D3 and the Canon 5D Mark 2. Both are excellent and both also have their strengths and weakness. I have posted my own personal reviews on Cnet on both of these camera systems. Regardless, even Nikon users, including myself openly admitted that Nikon had some serious catching up to Canon especially with respect to low noise and full frame technology. Kudos?s to Nikon for finally meeting market demands in these areas of camera technology. But in the real world, camera gear is still strictly a tool of the trade. In the right hands any relatively decent camera gear is more than capable of producing award winning photos. But, let?s refrain from all of this silly Brand trashing.

[Lakota01]

All I would like to say is that I have yet to see an Ansel Adams come out of the Digital DSLR age! All this instant gratification has seemed to breed nothing but a bunch of whinning, mines better than yours, keeping with the next Turkey rather than anyone that truly takes the time to want to learn all little ins and out and the true craft of using the light to produce the image with the equipment they purchase. Oh, but I forget myself we have Photshop and Lightroom and all those others to FIX what we can't do correctly in the first place. or even better yet it must have been the camera I better switch to that other brand I hear they better!!!! That will guarentee my pictures will be great!!! I have shot Canon film cameras, and Nikon film and digital D100, and currently D200, 300 and 700. As odd as it is I still have a 4X5 cherrywood view camera that will shoot circle around anything digital, but I still love all the digital world for the freedom from the chemicalsand yes the instant gratification. SO, Stop whining and just enjoy what you have and shoot the best pictures you can and always try to improve those pictures because you are never the best there is for long if you ever even are.