Speeds and Feeds

In part 1 and part 2 of this series, I claimed that there is apparently a secret rule in the microprocessor industry that determines the success--or failure--of new chip designs.

The failures included RISC processors, media processors, and intelligent RAM chips, which all sank in spite of clearly demonstrable advantages over alternative solutions. The great success is the programmable graphics processing unit (GPU), which has succeeded in spite of the sometimes wrenching shifts in programming methods and PC system architecture that have been required to support it.

So what's the secret? Simply this: a factor-of-two advantage, even if it's an inherent, persistent advantage, isn't enough to unseat an incumbent solution in the face of even the mildest competitive disadvantage. Without a factor of 10--a full order of magnitude--a new product won't even get a foot in the door.

That's why I call this rule the "factor factor." It isn't enough to be a few times faster than the existing alternatives. Given the performance consequences of Moore's Law, it's easier for your potential customers to wait a few years rather than spend a few years adapting to your "issues." You need be much faster than the products you're trying to replace. The target factor is 10--no less.

Sometimes, even a tenfold advantage isn't enough. One order of magnitude is enough to overcome one disadvantage, such as a change of programming methods. Add another simultaneous disadvantage, however, like the serious constraint in local memory capacity imposed by the IRAM concept, and the new technology may need a factor of 100 in performance to win a place in the market.

Overall, a new product must deliver net benefits amounting to as much as a full order of magnitude in cost, performance, or productivity to compensate for each significant disadvantage. That's just what it takes to motivate customers to deal with the problems rather than waiting for Moore's Law to speed up the solutions that are already familiar to them.

The introduction of the AMD64 instruction set by Advanced Micro Devices (also known as EM64T or "Intel 64" on Intel processors, or generically as x86-64) represents the ultimate success case for the factor factor.

AMD's Athlon 64 debuted the AMD64 instruction-set architecture.

(Credit: Advanced Micro Devices)

This isn't immediately clear, I suppose. Adopting the AMD64 standard required a lot of work by operating system vendors and software developers, and the performance benefit was relatively mild in most cases. But still, AMD64 was an immediate success because the performance benefit in certain applications--those that simply wouldn't fit into a 32-bit address space--was practically infinite.

Although the factor factor seems obvious--or at least it should--it's still at the heart of many failed products and hundreds of millions of dollars of wasted investments every year.

In Silicon Valley, like other chip-design centers around the world, projects rarely fail because of poor execution. In most projects, the engineers are good at their jobs, the managers are good at coordinating their work, and the investment is sufficient to get the work done.

Most projects fail at the conceptual level, before the detail design work even begins. The factor factor is only one of many reasons for these failures, of course, but it's the one that disturbs me the most because it's the easiest to anticipate.

This rule doesn't apply to all products. When a new chip for an existing market is architecturally compatible with previous products, a factor-of-two performance improvement is plenty. Even smaller benefits can justify the costs of developing a new product if there are few, if any, disadvantages associated with it.

Multicore CPUs are one of these products, at least for now. Process technology makes it pretty easy to double core counts. Dual-core CPUs were almost a drop-in replacement for single-core chips and caused no serious problems. Quad-core chips were the same thing again. Eight-core CPUs may be a lesson in diminishing returns, but I'm sure they'll be commercially successful.

Beyond that, we'll have to see how it goes. The critical advantage of the CPU over the GPU is high performance on inherently serial processing tasks (what we sometimes call "single-threaded applications"). On a typical PC, there's rarely more than a few of these tasks running at any given moment. It's always useful to have a few extra cores available for parallel tasks, but at some point (I'm thinking somewhere around the 16-core level), PC buyers are likely to stop paying extra for more extra cores.

Even mighty Intel could find itself on the wrong side of the factor factor. Given that quad-core chips became a mainstream product just this year, we can expect to see 16-core processors for ordinary desktop PCs in 2013 and laptops in 2015 or so. By that time, the GPU could be the incumbent solution for high-performance parallel processing, and multicore CPUs could be the technology looking for compelling performance advantages.

So...now you know the supposed secret. When you hear about a radical new microprocessor architecture, you can do what I do: imagine the numeral "1" followed by a "0" for each drawback you see in the proposal. Compare that figure with the claimed benefits and you'll know which way to bet.

By the way, kudos to CNET users divisionbyzero and TrinityTrident, who proved my point that this rule isn't really a secret by explaining it on their comments to the previous posts in this three-part series.

Now if someone could only explain why so many companies don't seem to know this rule!

Mobile data traffic is doubling every nine months, according to Cisco Systems. By 2013, mobile traffic will hit 2 exabytes--2 million terabytes--per month.

For some vendors, the growth rate is even higher. AT&T says its network load has been growing by 4.5x per year for the last two years, in large part (I assume) because of iPhone sales. You may have read about AT&T's pledge to spend over $12 billion this year to expand its wireless and broadband networks, including new 3G spectrum with better coverage and trials of 4G service.

At the Linley Group's Tech Processor Conference this week in San Jose, Calif., we learned what effect this growth is having on equipment makers, especially the companies making the microprocessors that go into network gear.

According to that same Cisco study, the problem goes well beyond iPhones. A 3G-equipped laptop "can generate as much traffic as 450 basic-feature phones" and 15 times the traffic of an iPhone or BlackBerry.

Networks have also gotten smarter, so network processors have much more work to do. Instead of just hundreds or thousands of clock cycles of work per packet on the network, new functions like firewalls, intrusion detection, and antivirus scanning to keep smartphones and laptops safe can require 100,000 cycles of processing on each packet.

Factoring in the growth in the network itself, Michael Coward of Continuous Computing, a company that sells equipment, software, and services to the telecom market, said that network operators need to achieve a 1,200x boost in processing performance between the systems deployed in 2008 and those that will be needed in 2013.

It's been a big week for small systems.

On May 29, VIA formally announced (here) its "Nano" family of low-power x86 processors. These chips will be especially valuable in small laptops, UMPCs, and so-called mobile Internet devices (MIDs).

Then on June 2, NVIDIA announced (here) its Tegra 600 family, which is also being marketed for MIDs. But Tegra is a very different animal. It's based on an ARM11 processor core, which can run Windows Mobile or Linux but not Windows XP or Vista.

VIA's Nano processor. The chip itself, the silver rectangle in the center, is about 7.7mm x 8.3mm.

(Credit: Courtesy of VIA Technologies, Inc.)

VIA's Nano processors are based on a new microarchitecture that is a giant step beyond previous VIA products and not far behind that of competing parts from AMD and Intel. Unfortunately, in this business, third place isn't a good place to be. VIA's older processors sold in relatively small quantities for low prices. Fortunately, they were very small and thus economical to make and sell.

The new Nano family offers much higher performance, with clock speeds from 1.0 to 1.8 GHz... but it's difficult to know what these clock speeds mean by comparison with AMD's or Intel's, and VIA isn't telling us, at least not directly. In this white paper on the Nano family, VIA only compares the performance of the new chips to its older C7 series.

But VIA does publish some numbers, so I was able to make some comparisons.

Take, for example, the Nano L2100 at 1.8 GHz vs. AMD's 2005-vintage Turion 64 ML-34 at the same speed, as found in the famous Acer Ferrari 4000 (reviewed here by PC World). The single-core ML-34 was much faster despite the clock-speed parity:

Of course, the ML-34 consumes much more power than VIA's processor; the ML-34 has a 35W TDP (thermal design power) specification, whereas the L2100 has a 25W TDP. The L2100 idles at a mere 500mW, but the ML-34 probably consumes at least ten times as much when idle.

To be fair, I'm not sure these are entirely fair comparisons, since VIA didn't publish the details of their system configuration. Also, VIA's performance position probably looks better on simple productivity applications, but I prefer to look at multimedia performance since that's what we usually find ourselves waiting on. It's been a while since we had to worry about out-typing our word processor...

I'm looking forward to seeing some good performance and power figures for Intel's Atom; I think the VIA chips will turn out to be effectively faster but run a little hotter. When I get more data, I'll post a comparison.

But considering that the Nano is generally 60% to 200% faster than the C7 and much more power-efficient than competing products from AMD and Intel, the new product family will likely improve VIA's market position significantly over the next year.

NVIDIA's Tegra, a high-integration processor for handheld gizmos such as mobile Internet devices.

(Credit: Courtesy NVIDIA Corporation)

NVIDIA's Tegra, on the other hand, offers no compatibility with existing PC systems or software, and its performance isn't even in the same class. The Tegra 600 family's ARM11 processor core runs at a maximum speed of 800MHz and, because it's a much simpler design, it offers a fraction of the effective performance of VIA's Nano.

So how can it possibly compete with Nano in mobile Internet devices?

Well, one answer is that Tegra is meant to deliver a much more complete solution with much lower power consumption. Instead of being just a core on a chip, like the Nano family, the Tegra 600 and 650 consist of a CPU core, a GeForce GPU, special-purpose hardware for accelerating digital video decoding and camera functions, and a dual-display controller that supports HDMI, LCDs, CRTs, and NTSC/PAL video. All of that on a chip the size of a dime, as you can see in the photo.

But the real answer is that what NVIDIA means by "mobile Internet devices" is different than what Intel (which coined the phrase), AMD, and VIA mean by it.

What NVIDIA means is basically any device with a size somewhere between that of a smartphone and a laptop, which can be used to access the Internet. But this doesn't strike me as a very useful definition; it boils down to encompassing anything like a smartphone with a larger screen. It's one thing to claim the Tegra 600 family supports a "full Internet experience" as NVIDIA did in advance briefings last month, but with the wide variety of sophisticated Web 2.0 websites out there, it really takes a PC-compatible system to deliver that experience.

Now, there's no doubt that the Tegra 600 and 650 will enable fun and interesting gizmos for people who buy lots of gizmos. (And honestly, I'm exactly that kind of person.) But I believe most people are not going to be interested in them. Anything larger than a cellphone is too big to carry around all the time. Anything with a screen smaller than about 7" to 9" isn't big enough for comfortable web browsing and movie watching. Anything with a screen that large might as well be a full Windows-compatible system.

Now, over time, these segments will inevitably blur together. Moore's Law will let us squeeze more performance into handheld devices. Software technologies like Adobe's Flash and Microsoft's Silverlight will allow more websites to work on simpler systems. Hardware like high-resolution LCDs and OLEDs and tiny projection displays will help solve size problems too.

But for now, I believe the Tegra 600 family is aimed at a market segment that isn't ready to develop, whereas VIA's Nano has a big market ready and waiting for it. The Nano won't sell as well as competing PC processors from AMD and Intel, but it should help raise awareness of VIA among PC buyers and encourage PC makers to keep pushing more functionality into smaller packages.

Second only to Moore's Law as a source of story ideas for pundits in the computer industry, Rambus was back in the news again last week.

This particular verdict was favorable to Rambus, but it wasn't the final word, nor was it exceptionally important. CNET News.com didn't even publish a news article about it, though Tom Krazit did write a pretty good blog post on the subject and it inspired a good post on intellectual property development from former Rambus exec Steve Tobak on his blog. Rambus has been involved in a great many lawsuits. Some of them work out in the company's favor, some don't, and I can't begin to predict what'll happen in the future.

I've written about Rambus many times, including this editorial for Microprocessor Report back in 2000. As I said there, I think Rambus should have disclosed what it was working on while it participated in the JEDEC standards organization.

But that's more a statement of ethics than law. JEDEC didn't require such disclosure at the time-- it does now!-- and other companies had allegedly done what Rambus did. Last week's court decision held that Rambus acted within the law at JEDEC, reinforcing the company's claims that it is owed licensing fees for its patents.

So I guess there are three things for which we should thank Rambus-- developing advanced DRAM technology, causing us all to think about the role of intellectual property in our industry, and providing work for professional bloggers.

Thanks, Rambus.

My friend Jerry Pournelle calls Unix the full-employment act for computer wizards (presumably a reference to the Humphrey-Hawkins Full Employment Act of 1978).

Similarly, I regard Moore's Law as the full-employment act for computer pundits. I've written about it several times myself (e.g. here and here); the phrase gets 930,000 hits on Google today.

One of the duties of any publication in the computer industry is to cast periodic doubt on the future reliability of Moore's Law, thus keeping the phrase prominent in the public perception. EDN Magazine discharged its duty for this year with great aplomb by publishing this piece last week.

You'll note this article says that this is "the first time" there's been such doubt. Never mind; they always say that.

The first time I heard that the sky was falling--excuse me, that Moore's Law was being threatened--was as the industry began to consider how to make chips with line widths below one micron (a millionth of a meter). That milestone was passed easily, and a while later we sailed past the quarter-micron mark, and now we're making chips with line widths a quarter of that--65 nm (nanometers, a billionth of a meter).

So now, right on schedule, doubt is being cast on our ability to reach the 15nm generation, which represents another four-fold linear reduction.

An experienced cynical analyst such as myself ought to just ignore this pronouncement, recognizing it for the latest iteration of a tradition that, after all, keeps guys like me in business.

But this time...I dunno, this time it seems more real.

In the past, the answer has always been the same: just draw thinner lines. This reveals new problems, which industry chemists and physicists have always solved in plenty of time.

But physics only goes so far. A 15nm-wide line in silicon is only about 30 atoms across. I'm willing to believe that a 30-atom-wide line will be stable enough. But how much smaller than that can we get?

Even if Moore's Law is safe this time around, the next four-fold reduction could well be impossible. And if that's when Moore's Law runs out--in silicon, at least--what am I going to write about then?

Have you ever heard of the Homebrew Computer Club? I'm sure you've heard of the products designed by its members: the Apple I and Apple II, the Osborne I, maybe even the earlier Sol-20 (one of the prettiest little personal computers ever; I have a beautiful example myself).

Wikipedia reports that the Homebrew Computer Club stopped meeting in "roughly 1977"-- about 30 years ago. But a small part of it survives. Some of the people in the Homebrew Computer Club spun off the Homebrew Robotics Club, and that club still meets regularly.

I try to attend meetings when I can, but I've been missing a lot of meetings since I took this job at Montalvo Systems. I missed the meeting this month; blogging about it sorta helps make up for that.

HBRC members are still mostly engineers and programmers. Some are parents and kids, which bodes well for the long-term health of the hobby as well as the long-term supply of workers for high-tech industries. Club projects involve everything from simple wheeled robots that can drive around on a tabletop without falling off the edge to GPS-equipped machines like small versions of the DARPA Grand Challenge vehicles. Pretty much every robot has a microprocessor brain; some have many microprocessors, each in charge of some subsystem.

The connection between microprocessors and robotics is not mere coincidence. Microprocessors revolutionized robot design. It was certainly a delayed effect, since microprocessors had to grow up for a while before they surpassed the previous methods for robot control. But when this happened, more or less in the 1990s, microprocessors put robots on the Moore's Law path.

No longer must robots remain tethered to industrial minicomputers, slaving away day after day to build Honda automobiles. In fact, Honda itself is making robots that can walk off the job if they want to.

Of course, they're not smart enough to want to, which brings me to the final ingredient required for intelligent, independent robots: synthetic brains. PC Magazine and CNET covered the recent Cognitive Computing conference, where researchers described projects to reimplement the brain in silicon. Not merely duplicate the results of thought, but the process itself.

I attended a presentation on this subject by Jim Burr at the 1993 World Science Fiction Convention in San Francisco. If I recall correctly, Burr projected that it should become possible to implement an electronic copy of the human brain, in a size that would fit inside a human head, consuming a similar amount of power, by 2050... and built in silicon, which switches far faster than biological neurons, the electronic version will be a thousand times smarter.

Moore's Law has probably slowed down a little since then, and it might slow down more, but it's still possible some of us will live to see this achievement. What happens then, I can't say, but maybe robots with silicon brains will start an Organic Humans Club...