This is a fascinating read from Baseline. I heard a bit about Hadoop and other Doug Cutting Lucene projects during a session at the O'Reilly Executive Radar session of OSCON last month. Hadoop is "an open-source project that aims to replicate Google's techniques for storing and processing large amounts of data distributed across hundreds or thousands of commodity PCs."
Sounds juicy, doesn't it? Especially in Yahoo's hands.
Tim O'Reilly gets this move exactly right: Yahoo is using open source in the Web 2.0 world in the same way that HP and other traditional software companies have used it in the packaged software world:
As a club to undermine competitors while blessing customers and developers.
(W)hy is Yahoo's involvement so important? First, it indicates a kind of competitive tipping point in Web 2.0, where a large company that is a strong No. 2 in a space (search) realizes that open source is a great competitive weapon against their dominant competitor. It's very much the same reason why IBM got behind Eclipse, as a way of getting competitive advantage against Sun in the Java market. (If you thought they were doing it out of the goodness of their hearts rather than clear-sighted business logic, think again.) If Yahoo is realizing that open source is an important part of their competitive strategy, you can be sure that other big Web 2.0 companies will follow. In particular, expect support of open source projects that implement software that Google treats as proprietary.
So here's the billion-dollar question: If Yahoo succeeds in commoditizing Google's secret sauce, will it matter?
The answer is likely "No," but that leads to the question I asked yesterday: If the source code doesn't matter, then why keep it closed?
The good news in all of this is that as Google, Yahoo and other Web companies seek to undermine each other by open-sourcing their competitors' perceived assets, everyone benefits in the form of more open source. IBM got involved in Linux to undermine its competitors, and we're all the better for this. (Well, except for Microsoft.) Sun open-sourced Solaris, Java, OpenOffice, etc. as ways to duke it out with its competitors, too.
It turns out the Web 2.0 world is no different. If anything, this trend should only move faster because operations and data, not source code and licenses, should matter more, making the decision to open up theoretically easier.
Open source, the highest stage of software capitalism.