An open-source problem? Too many scratches for too few itches

After writing my rebuke of Sidux, I came across an excellent post on a similar topic. Ryan Davis writes a cogent attack on software's tendency to reinvent the wheel. I heartily recommend that you read it.

While Ryan's critique lingers on open-source software, it's by no means limited to open source. In fact, the launch pad for his critique is a Microsoft project:

This problem is ingrained at Microsoft, which feels the need to brand everything, but it is in no way limited to them. A search on Sourceforge for "issue tracker" gives 585 results. Sifting through those to pick a winner is difficult.

It's more fun to write new code than read old code, but this fun wears off. After a certain initial momentum creating your new tool, you will inevitably come to a realization "this is going to take me for-[expletive]-ever". Unless your itch is particularly strong, you'll probably quit, and the world will be cursed with a 586th buggy issue tracker. By writing a plugin, you can ride the new-code high usually from start to finish, since its a much smaller task.

Nor is this limited to the vendor world. Red Hat's Jim Whitehurst recently argued that most of the world's software is built by enterprises for their own use...and then wasted in silo'd environments, if the code is used at all.

The proprietary world feels the need to rebuild everyone else's software in order to create walled-garden ecosystems. Unfortunately, the open-source world builds even more variants of the same products, though for different reasons.

Are we the world's least efficient market? Imagine what would happen if we could all pull behind a few credible alternatives, rather than inventing 585 of them?

[jrepenning]

Not "least efficient," no, because "efficiency" is benefit divided by cost. A part of the open-source juju is the low cost of failure: sure, there may be 585 issue trackers on sourceforge.net, but at least 583 of them cost only the volunteer time of some insufficiently motivated, quickly de-itched hacker. In an enterprise, you have to hire, resource, staff, equip, light, air-condition, parking-space, benefitize, and manage your insufficiently motivated, quickly de-itched hackers in order to get your failures. Much more expensive!

[chustar]

"Imagine what would happen if we could all pull behind a few credible alternatives, rather than inventing 585 of them?"
Yes, you're right. Let's pull behind windows as the de facto OS. MS Office as the office suite. Do you see a problem with believing we should all use the same thing? What if rather than develop their own search engine, the Google guys had simply pulled behind Yahoo!, an alternative to Ask and MSN.
What if Ubuntu was never developed, deciding instead to pull behind Debian, Fedora and a few other OSes? Would we have the user friendliness of Linux today? (poor as it is)

[royrusso]

"Imagine what would happen if we could all pull behind a few credible alternatives, rather than inventing 585 of them?"

Monopolies.

[rapier1]

Actually, its called the standards process. One of the problems with the standards process is that there are so damn many of them.

[TimBowden]

The answer is not to try and eliminate 580 odd alternatives, but to hide them. Letting 580 easily bored hackers build their own half baked never finished alternatives is by no means a bad thing. It does no harm whatsoever. Giving those extra 580 half baked solutions the same exposure as the top few is a problem. It should be extremely easy for average Joe to discriminate between the two classes of solution, or better yet, not even know the long tail of solutions is there unless they specifically go looking for it. Picking between two or three widely used and supported options is a reasonably doable task. Picking from 580 odd is not, unless you have very clear and specific needs that are not answered by the main choices.

[blabtech]

looks like another glitch...

http://blabtech.blogspot.com

[betelgeuse68]

There are many open source projects that are started by passionate people and taking a macroscopic view that their efforts are "inefficient" is a bit myopic (pun intended). The reality is, you can't direct people that in many cases aren't even being paid to work on projects that are deemed more valuable by you, me or industry in general. "Value" is highly subjective. They might be doing it simply because they enjoy writing code for a particular problem space and little else. Furthermore, open source developers aren't slaves. If they're not working on a project that is "dear" to them, they may not work on anything else. It's their time and therefore completely their prerogative.

[neilellis]

I have to say +1, that is entirely how I feel.

[Savio.Rodrigues]

Well said Roy!

[tuxperson]

Both Mr. Asay and Mr. Davis choose very poor examples to illustrate their points. Obviously Mr. Asay did not read the Sidux article- Sidux fills a need, which the article explains. Mr. Davis' errors were pointed out in the excellent comments; in a nutshell, there are good reasons for having multiple open source version control systems because each one meets different needs and solves problems with previous versions, and Microsoft's overriding strategy for everything they do is locking customers into their own software stacks, and propping up their monopolies. Not to mention their extreme allergy to genuine open source.

I wish that people like Mr. Asay, who have these wonderful bully pulpits at their disposal, would use them to deliver well-researched meaty opinions, instead of half-raw streams of consciousness. Yes, I may be envious of people who get paid to do the same thing that my friends and I do every day for free :)

[kkrugler]

Hi Matt,

I'd have to strongly agree with Tim Bowden - our solution at krugle.org to the problem of lots of half-baked projects is to (effectively) "hide" them by ranking projects such that these wind up on page 2 (or 22, or 222).

I'm a bit surprised that nobody has mentioned the essential Darwinian nature of open source. This to me is a key attribute, where survival of the fittest means it's OK to have lots of dead ends - as Jack pointed out, these are much cheaper in the open source model than what it would cost inside a company. And OSS projects tend to die quicker, while enterprise projects are more like dinosaurs that lumber along for years.

Finally, the quest for efficiency is often a fool's errand - by the time a top-down decision has been made about the "right" way to solve a problem, the landscape has changed, or somebody could have figured out a much better solution on their own and already released two versions.

This to me is the biggest problem with real adoption of the "open source way" inside of companies - you have to be willing to let projects quickly die off, and accept that things are messy in evolution...

-- Ken