Report: Open-source quality growing as it goes primetime

Given the vast and growing number of open-source projects, one would assume its quality had gone down as quantity went up. In fact, the inverse is true, suggests a new report from Coverity, which spent the past three years analyzing more than 11 billion lines lines of code from 280 open-source projects. This is crucial given open source's increased importance to the software industry as a whole, and not merely self-styled "open-source companies."

Among other findings, Coverity's report reveals a 16-percent reduction in static analysis defect density. While Coverity's analysis doesn't cover all or even most open-source projects, which number in the hundreds of thousands, it does tell us a great deal about the quality of the more successful projects like Linux, Firefox, Samba, and PHP.

Each of these projects is growing, and on average their quality is getting better. That's a feat of which few commercial software products can boast.

Such vendors are, however, taking notice. SAP, for example, despite its billions in sales, is trawling for sales leads on open-source start-up Openbravo's SourceForge.net project page.

SAP and other traditional software vendors aren't stupid. They can see a significant customer shift to subscription-based open-source offerings. Customers are increasingly looking for ways to lower costs and boost productivity through open source, as David Buckholtz, vice president of Enterprise Technology and Quality at Sony Pictures Entertainment, told the LinuxCon crowd Tuesday in a panel I moderated. Buckholtz suggested that what started out as a small experiment to replace BEA WebLogic, became a major shift to using open-source technology all over SPE, both to cut costs and improve product quality.

No, not all open-source software is fantastic, and undoubtedly even some of the commercial open-source software offerings are weak. The best open-source projects, as Intel's Dirk Hohndel pointed out in his LinuxCon keynote, are those with strong execution and vision. Just like in the proprietary software world.

Coverity's analysis, however, suggests that open-source software may have the upper hand on its proprietary peers. Open-source quality is almost certainly a direct result of open-source transparency, something Red Hat CEO Jim Whitehurst suggested at Red Hat Summit recently when he opined, "If we all had to walk around naked we'd all spend more time in the gym."

An open-source project will only be as good as the developers who work on it, but those developers have a strong motivation to make the code secure, robust, and high performance. The code is "naked," as it were. The source code is open.

Customers and competitors are noticing.

Disclosure: SAP Ventures is an investor in Alfresco, my employer, and I am an adviser to Openbravo.

[jrepenning]

The prnewswire article seems to spin this differently: Coverity monitored some OSS projects over time, filed bugs on what they found, and observed that these bugs got fixed. That wouldn't seem to say anything at all (good or bad) about projects not under Coverity monitoring.

It also strikes me as a bit circular: unsurprising that the class of bugs that get filed matches the class of bugs that get fixed. Coverity would impress me more if they could show that their monitoring and reporting correlates with reduced user-reported defects or downtime.

[twistedbiscuit]

Showing a correlation with reduced field-reported defects or downtime requires developing the same software both with and without the use of Coverity. How are you going to do that? The best you can realistically do is find how many field-reported bugs *would have* been found had Coverity been used during development. That requires a large effort that few open source developers have the time or inclination for.

[jrepenning]

You're right, that showing the stats that mean something is hard. My point is, we shouldn't claim the result we agree we can't collect.

[jaguar717]

Hate to be the negative nancy, but I just keep reading the same article over and over again. Just like fans of hopeless sports teams love to say, THIS is going to be the year.

I've been reading articles about open source finally hitting "the big time" for as many years as I can remember--and I'm only 23.

To get any non-negligible share of the market you need to be selling a 100% complete service where EVERYTHING behaves and you cover ALL of the extra time involved in getting open source to work, or your customers will throw their hands up and buy commodity software. Of course, once you've invested all those resources to get everything tweaked and customized, you tend not to want to just give that away, so it's not really that "open" any more.

[plugtree]

As an open source committer I can say our motivation is different from commercial programmer, we code to be proud about our product and everyone can see your discipline, creativity and/or code elegance. I think commercial programmer have different motivations, have to deal with other stuff, etc. We code because we love to.

[bigskia]

Open source may appear to be the most cost effective, but generally speaking (in the long run) it becomes a time consuming, frustrating process as you try to adapt the software to your needs.

Don't waste valuable man hours in development, instead use reporting solutions like Windward Reports that already work.