Linux to get mainstream server tests

A consortium devoted to improving Linux for high-end servers will announce on Monday a suite of tools designed to make widely used speed tests available to Linux programmers.

The Open Source Development Lab (OSDL) has released three speed tests, or benchmarks, that correspond to three tests from the Transaction Processing Performance Council (TPC). The tests are subsets of benchmarks from the TPC consortium that measure a server's ability to handle an inventory tracking system, extract useful information from large data sets, and run a large online bookstore.

Benchmarks are important for customers who want to find which computer systems give them the most bang for their bucks. But benchmarks also are important for programmers trying to evaluate what changes are a good idea.

And speed tests are particularly important in Linux. Linus Torvalds, founder and leader of the collaborative programming project, has fostered an environment in which modifications are much more likely to be considered and accepted if they're submitted with accompanying proof that they speed up a system.

The new benchmarks are higher-level tests that better measure overall system performance, said Tim Witham, director of the OSDL. Some benchmarks used by programmers working on the heart, or kernel, of Linux measure only one facet such as disk throughput or the speed with which the computer can switch from one task to another.

The OSDL benchmarks may be downloaded and used for free. The benchmark software is covered by the Artistic License, which permits modifications but requires a different name if changes are major.

Linux increasingly is developed by programmers at corporations such as IBM, Red Hat, Hewlett-Packard and Intel--all sponsors of the OSDL--who have been able to run internal tests but have been prohibited from sharing them with the Linux community at large, Witham said.

"In the Linux world in particular, (a benchmark result) needs to be shared without restriction. You can't wait for someone to approve of it," he said. "You have to have something you can do in the near real-time of Linux kernel development."

Two OSDL employees worked for a year to create the three benchmarks, Witham said. The tests correspond to the TPC-C, TPC-H and TPC-W tests from the Transaction Processing Performance Council.

TPC-C in particular is widely watched, though companies willing to spend millions of dollars on hardware and tuning are more likely to achieve better results. Long the domain of Unix servers using RISC processors, the TPC-C test now has been infiltrated by a new generation of Windows servers and Intel processors.