February 4, 2005 12:35 PM PST

Study: Few bugs in MySQL database

A source-code analysis of the MySQL database, a popular open-source program at the heart of many Web sites, revealed few bugs compared with the number found in commercial code, testing company Coverity said Friday.

The analysis, done using the company's homegrown tools, found 97 flaws, at least one of which was a serious security problem, Coverity said in a report. However, that number is small compared with most commercial software code, said Seth Hallem, Coverity's CEO.

"It's like if you get ready to go to your high school reunion, you probably work out a bit before you go."
--MySQL's Zack Urlocker,
on the incentive to make
sure open-source code is clean

"In terms of industry averages, MySQL is excellent," Hallem said. "There is not a lot of easy gotchas in there."

Source-code analysis tools such as Coverity's are quickly becoming must-haves for software developers. Microsoft uses its own internal tools to vet its software, find bugs and reduce security vulnerabilities. Other companies, such as Ounce Labs and Reflective, have sold their wares to major companies. Coverity counts technology giants Cisco Systems and Oracle among its customers.

MySQL, the Swedish company that develops and maintains the MySQL database, contacted Coverity and asked for the audit, said Zack Urlocker, vice president of marketing for MySQL.

"We have fixed all the bugs that have been reported," Urlocker said. "And they will go out in our next release."

While the analysis software does not catch all bugs, the programs can effectively find certain classes of software problems. In many cases, such flaws could be the low-hanging fruit that might otherwise be found by an external hacker or independent security researcher. Moreover, since many companies allow free use of these tools for noncommercial software, an open-source project will likely have to analyze their code or risk attacks by malicious attackers who use the tools first.

Eliminating bugs is not the only use of such tools. Many IT professionals look to analysis tools to generate a measure of the quality of two code bases for comparison. While open-source software has its own share of problems, the fact that MySQL has fewer than 100 bugs indicates that the open-source database has been well-coded, Hallem said.

"By eliminating these, we are eliminating the most obvious flaws in the code," Hallem said.

Commercial code typically has anywhere from one to seven bugs per 1,000 lines of code, according to an April report from the National Cybersecurity Partnership's Working Group on the Software Lifecycle, which cited an analysis of development methods by the Software Engineering Institute at Carnegie Mellon University.

Coverity's analysis of MySQL found an average of one bug in every 4,000 lines of code--results that are at least four times better than is typical with commercial software.

The findings parallel earlier work by Coverity in auditing the Linux kernel; that work found that a recent version of the kernel had 985 flaws in 5.7 million lines of code, less than a single flaw in every 10,000 lines of code.

"It is similar to other studies that have been done in the past that have shown that open-source code is clean and well-structured," said MySQL's Urlocker. He added that the open-source development process compels programmers to write cleaner code because the code will be seen and evaluated by others.

"It's like if you get ready to go to your high school reunion, you probably work out a bit before you go," he said.

By analyzing Linux and MySQL, Coverity has done quality checks on two of the four common components of open-source-based Web servers. The other two components--the Apache Web server and the PHP Web-scripting language--will be analyzed in the near future, Hallem said.

11 comments

Join the conversation!
Add your comment
German?
"MySQL, the German company that develops and maintains the MySQL database"

MySQL is a Swedish company based in Uppsala.
Posted by (1 comment )
Reply Link Flag
"Only?"
Try sell a car with "only" 97 flaws and see if ANYONE will buy your car. Give me a break.
Posted by 201293546946733175101343322673 (722 comments )
Reply Link Flag
they do sell cars with problems!!
Hey Bob u ever looked through a dealership recall/service manual for a car?? i have seen the dealership manual for a 'big' car company and they had well over 97 known problems on every one of there cars!! - they just dont make it public!!!!
Posted by (3 comments )
Link Flag
Windows has thousands of bugs
And it is good enough for you
Posted by Bill Dautrive (1179 comments )
Link Flag
It's far fewer than anything out there....
MySQL is the world's leading database system out there, and you don't see it getting hacked to death.

This article is intentionally (and correctly I might add) implying that all other databases have far more than 96 flaws!
Posted by hion2000 (115 comments )
Link Flag
Inconclusive
As we all should know by now, "flaws" are not limited to mistakes in coding. This label is also used for mistakes in logic that only become apparent through the use of an application. If you ask me, refusing to support stored compiled queries is a flaw. But there are certainly users who will go beyond the intent of the developers and discover real problems that no code analysis would have found.
Posted by David Arbogast (1709 comments )
Reply Link Flag
True
...and what may be considered a flaw by some people might have been intended by the developers, or vice versa.
Posted by hion2000 (115 comments )
Link Flag
It is not that simple
Complex systems will always have flaws. It is a
matter of physics. If one patches a flaw it may
introduce another flaw. Even if a system is stable
it will inevitably fail. The second law of
thermodynamics sates this clearly. In a larger sense,
all things tend toward chaos. I don't think you
would ever use an airliner if you knew how many
flaws existed in it (known and unknown flaws). But,
as many studies have stated, the open-source
development model produces much better quality.
The reasons are varied, but, the primary one is
human nature. When submitting something that will
be evaluated by a large group, one tends to do
your homework before committing.
Posted by Johnny Mnemonic (374 comments )
Reply Link Flag
known and unknown flaws
<a class="jive-link-external" href="http://www.analogstereo.com/jaguar_xj6_owners_manual.htm" target="_newWindow">http://www.analogstereo.com/jaguar_xj6_owners_manual.htm</a>
Posted by George Cole (314 comments )
Link Flag
This is silly
I love it when people compare MySQL to Oracle or MS SQL Server. I have no doubt that MySQL's long term goal is to offer a truly enterprise capable RDBM, but as of today they don't have it. However since we're on the subject I would like Oracle and Microsoft to put their code up to the test. I have a feeling they are both going to have more than 97 errors. Or if you like they are going to have more errors per every thousand lines of code.

For what MySQL is today it is a very capable RDBM. It lacks a lot of what enterprise user would need, but is a well developed application. A lot can be said for Oracle and SQL Server. They are two very powerful RDBM's.

True enough this report probably is an accurate real world simulation of flaws. I can't believe the way some have treated this though. They act like 97 flaws in MySQL is unacceptable. How many flaws do you think the other would have?
Posted by System Tyrant (1453 comments )
Reply Link Flag
 

Join the conversation

Add your comment

The posting of advertisements, profanity, or personal attacks is prohibited. Click here to review our Terms of Use.

What's Hot

Discussions

Shared

RSS Feeds

Add headlines from CNET News to your homepage or feedreader.