November 28, 2001 2:00 PM PST
IBM unseated in new supercomputer ranking
IBM has kept its place atop the last three versions of the Top500 ranking of the world's fastest computers. But Big Blue has been demoted on a new list from IDC that the research firm says better represents real-world performance.
At the top of the new list, created by IDC in cooperation with supercomputer users, is a 3,024-processor Compaq Computer machine called Terascale based at the Pittsburgh Supercomputing Center.
An IBM system called ASCI White, No. 1 on the earlier list, is No. 2 on the new list. NEC computers take third and fourth places, with a Cray machine in fifth.
Cray took top honors when it came to the largest presence on the new list. Though IBM had the largest number of computers on the old list, it has less than half of Cray's count on the new list.
Cray had 58 of the 149 systems in the "capability class" of supercomputers--the most powerful and expensive models in the IDC study--or about 39 percent. IBM and SGI each had 24 systems, or about 16 percent. Compaq had 15 systems, NEC had 12, and Fujitsu had eight.
In the "enterprise computers" class, a lesser category for machines costing $1 million or more, Cray also came out on top, with 43 of the 95 systems. Compaq was second with 14, and Hewlett-Packard was third with 13.
SGI tops the list for systems costing $250,000 to $1 million, and HP leads for those under $250,000.
The new list grew out of dissatisfaction with how well the Top500 list represented actual computing power. The current Top500 list is based on the Linpack measurement, which reflects how well processors work but excludes other computing qualities, such as how fast data can be transferred from one part of the system to another. Linpack thus reflects peak performance but not real-world results.
In 2000, the Top500 organizers joined IDC and researchers from the government, industry and academia to come up with a more useful benchmark.
The new benchmark includes Linpack and the SPEC FP benchmark to measure a CPU's mathematical prowess. A second component includes the Stream measurement of how fast a system can transfer data to memory. A third combines a measurement of the total number of CPUs and the combined memory transfer capacity.
"The excessive focus on peak performance has contributed to a tendency for high-performance computing (HPC) vendors to develop new generations of supercomputers showing impressive gains in peak performance while lagging behind in many critical system features," IDC analyst Debra Goldfarb said in a statement. "This has also made it difficult for some HPC users to justify purchasing computers that may provide substantially better performance and value in practice."
IDC and the HPC Users Forum also are working on a set of application-specific performance measurements. That set of benchmarks has not been released.