The factor factor, part 3

In part 1 and part 2 of this series, I claimed that there is apparently a secret rule in the microprocessor industry that determines the success--or failure--of new chip designs.

The failures included RISC processors, media processors, and intelligent RAM chips, which all sank in spite of clearly demonstrable advantages over alternative solutions. The great success is the programmable graphics processing unit (GPU), which has succeeded in spite of the sometimes wrenching shifts in programming methods and PC system architecture that have been required to support it.

So what's the secret? Simply this: a factor-of-two advantage, even if it's an inherent, persistent advantage, isn't enough to unseat an incumbent solution in the face of even the mildest competitive disadvantage. Without a factor of 10--a full order of magnitude--a new product won't even get a foot in the door.

That's why I call this rule the "factor factor." It isn't enough to be a few times faster than the existing alternatives. Given the performance consequences of Moore's Law, it's easier for your potential customers to wait a few years rather than spend a few years adapting to your "issues." You need be much faster than the products you're trying to replace. The target factor is 10--no less.

Sometimes, even a tenfold advantage isn't enough. One order of magnitude is enough to overcome one disadvantage, such as a change of programming methods. Add another simultaneous disadvantage, however, like the serious constraint in local memory capacity imposed by the IRAM concept, and the new technology may need a factor of 100 in performance to win a place in the market.

Overall, a new product must deliver net benefits amounting to as much as a full order of magnitude in cost, performance, or productivity to compensate for each significant disadvantage. That's just what it takes to motivate customers to deal with the problems rather than waiting for Moore's Law to speed up the solutions that are already familiar to them.

The introduction of the AMD64 instruction set by Advanced Micro Devices (also known as EM64T or "Intel 64" on Intel processors, or generically as x86-64) represents the ultimate success case for the factor factor.

AMD's Athlon 64 debuted the AMD64 instruction-set architecture.

(Credit: Advanced Micro Devices)

This isn't immediately clear, I suppose. Adopting the AMD64 standard required a lot of work by operating system vendors and software developers, and the performance benefit was relatively mild in most cases. But still, AMD64 was an immediate success because the performance benefit in certain applications--those that simply wouldn't fit into a 32-bit address space--was practically infinite.

Although the factor factor seems obvious--or at least it should--it's still at the heart of many failed products and hundreds of millions of dollars of wasted investments every year.

In Silicon Valley, like other chip-design centers around the world, projects rarely fail because of poor execution. In most projects, the engineers are good at their jobs, the managers are good at coordinating their work, and the investment is sufficient to get the work done.

Most projects fail at the conceptual level, before the detail design work even begins. The factor factor is only one of many reasons for these failures, of course, but it's the one that disturbs me the most because it's the easiest to anticipate.

This rule doesn't apply to all products. When a new chip for an existing market is architecturally compatible with previous products, a factor-of-two performance improvement is plenty. Even smaller benefits can justify the costs of developing a new product if there are few, if any, disadvantages associated with it.

Multicore CPUs are one of these products, at least for now. Process technology makes it pretty easy to double core counts. Dual-core CPUs were almost a drop-in replacement for single-core chips and caused no serious problems. Quad-core chips were the same thing again. Eight-core CPUs may be a lesson in diminishing returns, but I'm sure they'll be commercially successful.

Beyond that, we'll have to see how it goes. The critical advantage of the CPU over the GPU is high performance on inherently serial processing tasks (what we sometimes call "single-threaded applications"). On a typical PC, there's rarely more than a few of these tasks running at any given moment. It's always useful to have a few extra cores available for parallel tasks, but at some point (I'm thinking somewhere around the 16-core level), PC buyers are likely to stop paying extra for more extra cores.

Even mighty Intel could find itself on the wrong side of the factor factor. Given that quad-core chips became a mainstream product just this year, we can expect to see 16-core processors for ordinary desktop PCs in 2013 and laptops in 2015 or so. By that time, the GPU could be the incumbent solution for high-performance parallel processing, and multicore CPUs could be the technology looking for compelling performance advantages.

So...now you know the supposed secret. When you hear about a radical new microprocessor architecture, you can do what I do: imagine the numeral "1" followed by a "0" for each drawback you see in the proposal. Compare that figure with the claimed benefits and you'll know which way to bet.

By the way, kudos to CNET users divisionbyzero and TrinityTrident, who proved my point that this rule isn't really a secret by explaining it on their comments to the previous posts in this three-part series.

Now if someone could only explain why so many companies don't seem to know this rule!

[CrazeeBrit]

I can think of a much better name than the lame "factor factor." I think you should name your theory "Order Of Magnitude Factor" or OOMF. In order for your technology to take off, you need OOMF.

[Peter N. Glaskowsky]

That's very good! Thanks. :-)

[sparrowhyperion]

So many companies don't seem to know this rule because nowadays, most of them are run by Bean Counters who don't have a clue, and not by scientists, technicians, or engineers. Lose the bean counters, or at least put design decision completely in the hands of the engineers, and you will see less money and time wasted.

[Peter N. Glaskowsky]

You might be surprised by how many companies-- good and bad-- are run by the architects and engineers. I think that well over half of the 100+ 3D, network processor and extreme processor companies I studied at Microprocessor Report were founded and led by technologists.

. png

[Shankland]

I wonder to what extent the arrival of rival processors--however doomed in advance--apply the necessary competitive pressure to the incumbent technology to keep it moving ahead. Perhaps it would save billions of dollars not to introduce processors that don't have the necessary advantage, but would there be other costs elsewhere in the system because incumbents would be complacent?

[Peter N. Glaskowsky]

That's a good point. As I mentioned, CISC processors adopted RISC-like internal pipelines to allow them to run faster, and the media-processor industry died when Intel and other CPU vendors adopted SIMD extensions. But I think it's likely that those changes would have happened, and on roughly the same schedule, without the costly attempts to push RISC and media processing into the consumer market.

. png

[SomeoneWhoShouldKnow]

Good point. Intel and AMD would have adopted the RISC concepts less due for competitive reason, but more because they were just really good microarchitecture ideas. AMD's Mike Johnson wrote the book on x86/RISC design before building the ill-fated K5 (which was internal RISC).

It's not the bean counters that's the problem - it's the engineers that so want to build their next big idea, that they bamboozle the beancounters into thinking they can develop a sustainable advantage over the incumbents and deliver the project on schedule.

PowerPC did succeed to some extent in that it's still the core for all the game consoles.

[epobirs]

This reminds me of why I told people that Ageia was doomed as a maker of standalone products. Getting direct support for their physics acceleration product was a serious uphill chicken/egg battle. Without a substantial installed base, what developer was going to make the large capital investment to produce a must-have game that couldn't be executed without the Physx card for a substantial period of time after the game's release.

The Physx card's boost wasn't the OOMF needed to make that game unique for long enough to matter, supposing it could be reliably produced at all. After all, it needed to be a great game and not just a great demo of the chip. Capital and desire have often fallen short of delivering success in that field.

The other problem faced by Ageia and many others, is how easily can your idea by reduced to a subsystem in the incumbents' products? If you don't have that OOMF going for you, chances are it will present no great challenge to the incumbents' engineers after you've gone to the trouble and expense of identifying the value of the idea.