The Open Road

Desperate for a deal after sleeping right through Wal-Mart's early-morning Black Friday frenzy? You're in luck. The best deal this holiday season may be just a download away.

That's right: Handbrake, arguably the world's best video transcoder, just hit version 0.9.4.

And boy, is it beautiful.

Handbrake has long been my go-to choice for ripping DVDs to my hard drive (saves battery life when watching videos while traveling and ensures my kids won't ruin the DVDs), but this particular version exceeds my expectations. Why? Because it delivers over 1,000 new enhancements while delivering better picture quality at a smaller file size and faster.

Or as the Handbrake developers say:

There's an old proverb in the video encoding world: "Speed, size, quality: pick two." It means that you always have to make a trade-off between the time it takes to encode a video, the amount of compression used, and the picture quality. Well, this release of HandBrake refuses to compromise. It picks all three.

This isn't hype. In my own use of the software during the past week, performance is noticeably faster, and picture quality is awesome.

Importantly, while the Handbrake developers have been hard at work over the past year to update the venerable video transcoder, the team owes a lot to developers from the x264 project:

A large portion of these speed, size, and quality improvements come to us for free, from the x264 project. The past year, like every year, has seen some massive improvements for that video encoding engine. As always, it has been further hand-optimized for better performance. But it has also gained new features like macroblock tree rate control and weighted P-Frame prediction.

This is how open-source development works: Handbrake focuses on what it does best (User interface, features like live preview, etc.) while leveraging the best of other project's strengths.

It's a recipe for a supereasy and very powerful transcoding experience. And at a 100 percent discount now through forever (Handbrake is open source and costs nothing to download), now is a good time to download it and let 'er rip, whether you run Mac (Intel 32-bit and 64-bit, plus PowerPC), Linux, or Windows.

When an open-source project is working optimally, can proprietary-software companies hope to compete?

Greg Kroah-Hartman, a prominent Linux kernel developer and Novell fellow, suggests that the answer is no. Speaking to the How Software Is Built blog, Kroah-Hartman makes the case that the pace of Linux development leaves competition in the dust:

[The Linux kernel development team adds] 11,000 lines, remove[s] 5,500 lines, and modif[ies] 2,200 lines [of code] every single day.

People ask whether we can keep that up, and I have to tell you that every single year, I say there's no way we can go any faster than this. And then we do. We keep growing, and I don't see that slowing down at all anywhere.

I mean, the giant server guys love us, the embedded guys love us, and there are entire processor families that only run Linux, so they rely on us. The fact that we're out there everywhere in the world these days is actually pretty scary, from an engineering standpoint. And even at that rate of change, we maintain a stable kernel.

It's something that no one company can keep up with. It would actually be impossible at this point to create an operating system to compete against us. You can't sustain that rate of change on your own.

Microsoft might beg to differ, as would Apple, but the reality is that neither is updated as often or as extensively as Linux is, which supports a far broader hardware portfolio than any other operating system in existence.

Linux is pretty incredible. But it's not alone. Mozilla Firefox, Eclipse, and other projects produce best-in-class software at an almost frightening pace.

Can anyone compete with an open-source project at the top of its game?

The answer might well be no, as the top open-source projects are collaborative efforts between multiple companies that pool resources and expertise to drive development. And while it might seem reasonable that a single corporation could best open source's seeming "development by committee" approach, the reality is that well-managed open-source projects have none of the inertia that one might expect from a communal approach.

Quite the opposite.

Having said that, very few open-source projects actually meet the criteria that enable Linux's success. Most appeal to a too-narrow and too-small population of developers (i.e., single-company projects) to glean the benefits and scale of Linux-like development.

As such, the proprietary-software companies probably won't have to worry about competing with indomitable open-source competitors. Not most of the time, anyway.

For those that do, however, better stock up on the pumpkin pie. It may be the only thing to be grateful for this Thanksgiving season.

Greg Kroah-Hartman interview discovered via @glynmoody's ComputerWorld blog.

Life has never been better for enterprises and consumers. From free music to free software, the digital economy is an all-you-can-eat free-for-all.

That is, unless you're a vendor.

Traditional vendors are getting shellacked by the digital economy, spurring some, like Rupert Murdoch and his News Corp., to threaten to stick a finger in the dike and demand that users pay for content. (At Murdoch's Wall Street Journal, users already do pay to access some stories online.)

The problem with this approach is that not everyone is willing to follow suit. Why? Well, not everyone needs to. The BBC responded to Murdoch's plans by declaring it won't charge for content. It doesn't need to. U.K. taxpayers already fund it.

Different strokes for different folks. And different business models, too.

Google makes money by making it easy to discover others' content. So does Apple's iTunes. Google can afford to give away lots of free software (and even free hardware) to nudge people into its advertising model.

That's hugely disruptive.

In software, Microsoft doesn't like competing with free Linux. Microsoft spends a lot of money developing Windows. It must seem unfair to have to compete with the rest of the industry, which increasingly coalesces around Linux (or Android, or MySQL, or...).

But that's life in the open-source economy. Your core competence is always going to be someone else's throwaway complement, and ripe for open-source commoditization.

In fact, it may be getting worse, and not just for Microsoft. The Wall Street Journal reports that Domino's Pizza has rolled out a multimillion dollar, homegrown pizza-ordering/fulfillment system.

Could Domino's have bought an off-the-shelf system from Oracle, SAP, or another vendor and customized it? Probably. But then, this isn't how most IT gets built, anyway.

Most software is written by enterprises to use, not for sale, as Bruce Perens and others point out. So while we credit Microsoft, Oracle, and others as the backbone of the "software industry," the reality is that these companies are really a drop in the software bucket, with companies like Sony, Wal-Mart, and GE the true backbone of a much larger software ecosystem than the vendors comprise.

As open source matures, we're going to see these "software users" develop more software in-house, often building from open-source projects. Gartner calls out intriguing proof of this trend, but it's equally evident in anecdotes like this one, highlighting Virgin America's adoption of open source to reduce costs and improve innovation.

Virgin America is writing few checks to external vendors. That money is paying internal developers instead.

Digitization, then, may not be destroying the software market so much as reshaping it. In this new model, companies like Domino's will need more internal developers as they rely less on outside software vendors.

There will still be a need for companies like SAP, of course, as there are broad industry needs that a company or open-source foundation can satisfy. But for strategic IT projects, we're likely to see more open source plus internal development, and less packaged software purchases.

If something seems too good to be true, it probably is. That popular aphorism never seemed truer than today when reading The Wall Street Journal's analysis of Wikipedia's declining volunteer base. Despite countless articles extolling the virtues and seeming omnipotence of "community" over the past several years, the technology industry seems to be settling back into old habits:

Command and control.

It's not that the "wisdom of crowds" idea hasn't influenced the way technology is developed, or how news and information are gathered and distributed. It has.

It's just that the promised sea change has proved to be far less disruptive than we expected.

Take Wikipedia. As the Journal calls out, volunteerism has declined as the ease of contribution has waned. The easy topics are taken. Rules for upping the quality have proliferated. Wikipedia is becoming...corporate.

Nick Carr has been pointing this out for years, but it's only now becoming self-evident. Wikipedia has grown up and, in so doing, is looking more and more like the encyclopedic world it sought to displace.

Nor is it alone. Open-source business models increasingly look like proprietary software models, as the Software Freedom Law Center's Bradley Kuhn suggests.

Even uber successful open-source communities like Joomla have discovered that reliance on volunteers falls short of what a few good paid developers can do.

That's a positive discovery by Joomla. A more worrisome discovery is that Mozilla remains far too dependent on Google to fund development of Firefox. Mozilla has lots of community, right? Yes. As Mozilla CEO John Lilly has said, 40 percent of Firefox's code comes from developers not employed by the foundation.

But that still leaves 60 percent, and virtually all of the core development work, that relies on "company," not "community," which is how much of the world's best open-source software is developed: funded by IBM and other "community" members.

For those who think "community" is a euphemism for "everyone else doing my work for me," think again. It just doesn't work that way.

Of course, companies can go to the opposite extreme, too. Apple, for one, gets beat up for a heavy-handed approach to its App Store approval process. Apple, in other words, doesn't seem to care one iota what "the community" thinks.

But then, this is the same App Store with more than 100,000 applications and 2 billion downloads to date. No wonder Apple isn't apologizing: it's clearly benefiting most people most of the time, or the application developers would take their complaints to a different platform.

But they haven't, and this calls out the problem with deifying "community." It's accepted wisdom that one shouldn't "anger the community," as if it's some unknown god that demands the occasional virgin to be thrown into the volcano. But the truth is, "community" is not really much different from the "customers" and "partners" the industry has sought to satisfy for decades.

So, yes, by all means seek to work with your community of users and partners, but don't expect "the community" to do your work for you. Guess what? "The community" already has a day job, and can't afford to work full-time for you unless you pay it.

All of which leaves us largely where we started. The most successful software companies don't rely on some vague "community" to build their products. Microsoft, Oracle, IBM, Google (Android, anyone?), and even, increasingly, Red Hat (JBoss, KVM, etc.) build great software based on their own, internal plans and expertise and "the community" buys it (or resells/embeds/etc. it).

The big shift, however, has been in the transparency of the feedback loop, which has been a welcome change in the industry. So, to the extent that "community" simply implies a more open way of developing and distributing software, then, yes, it has been significant.

But it hasn't changed the world. It has only changed the way the dominant technology companies...dominate.

For those who have fret about Microsoft fighting against open source, I have news for you: Microsoft's impact on open source may be worse as a friend than as an enemy.

Over the past few years, Microsoft has steadily warmed to open source, to the point that it now hosts its own open-source code repository and has seen its Microsoft Public License used more often than venerable licenses like the Mozilla Public License or the Eclipse Public License, according to new data released by Black Duck Software.

The open-source world should be worried.

After all, as IBM's Savio Rodrigues points out, an open-source-friendly Microsoft no longer has qualms about embedding open-source software like MySQL into its products. In particular, Microsoft supports MySQL as part of its Azure cloud service...without paying Sun a dime for the privilege.

It's a completely legitimate way to offer open-source value to Microsoft customers, and is very similar to what Amazon is doing with MySQL.

However, as Rodrigues notes, it's not necessarily good for MySQL, or other open-source projects that could be used this same way:

The larger point is if Amazon, Microsoft, IBM, HP, Google, Cisco, EMC/VMware, or Oracle/Sun offer a simple and supported cloud service for running MySQL, Tomcat, JBoss, Mule, or Apache HTTP instances, what reason do customers have to acquire "enterprise subscriptions" from the vendors developing these open source projects? Until now, the value of an open source "enterprise subscription" has largely been access to support and access to administration and management tooling. In the case of MySQL, the former is provided by Amazon RDS and Azure SQL as part of the per-hour service. Again in the case of MySQL, the latter is rendered unnecessary or replicated through Amazon RDS and Azure SQL tools.

Consider it a super-friendly, and super-dangerous, bear hug.

For those who think that this affects commercial open source and not community-led open source, think again. Money and open source don't grow on trees.

The explosion of open-source development has directly correlated to the explosion of cash investments into open-source projects, starting with IBM's $1 billion commitment to Linux. MySQL, the database, would be a pale shade of what it is today without MySQL AB, the company that has funded the overwhelming majority of its development.

So, is this cause to castigate Microsoft? No. After all, it's really no different from what Amazon, Google, Apple, and others do with open source.

Rather, Microsoft's move should serve as a reminder to open-source companies that they need to upgrade their business models or risk being rendered irrelevant by the cloud and all that it enables vendors to do with open-source software.

After all, the protections that the GNU General Public License (GPL) and other open-source licenses offer in the traditional software world are essentially meaningless in the networked world, where software is used to create services, but isn't actually distributed.

This is as true for Red Hat as it is for open-source start-ups like Openbravo and Talend. Imagine if Amazon decides to start offering JBoss as a cloud service. Or Red Hat Enterprise Linux, for that matter (minus the trademarks).

It could happen. Actually, I'll go one step further: it will happen. It's just a matter of when.

This is why companies like IBM, Google, and increasingly Microsoft strategically invest in open source, but don't try to directly monetize open source. It's also why the "open-source companies" need to figure out a Plan B before Plan A gets taken from them.

Government policies favoring open-source software adoption should be wildly popular within the open-source crowd. Yet, at an open-source conference in Amsterdam today, I kept hearing the opposite. Despite the Dutch government's best intentions to foster open-source adoption, some people think it may actually be doing the opposite.

By many measures, the Netherlands is a great place for open-source software. In 2007, the government started to phase in a policy that gave preferential treatment to open-source software in IT purchasing decisions. Initially, at least, the policy seems to have been a success, with a July 2009 study highlighting a wide array of open-source software in use by government.

Sounds good, right?

Maybe not. According to sources within the government and others that sell to the government (both proprietary and open-source vendors), the government's rigid definition and management of the policy has more often than not thwarted its attempts to go open.

At its core, however, the problem derives from a mismatch between ends and means. The government's goal--"to increase the sustainability of information and innovation, while lowering costs through the reuse of data"--is not always best achieved by open source. A proprietary program with a broad community that is fully open standards-based could actually be a better solution to achieve this end than an open-source solution, particularly if it has a small community and smaller adoption.

That's because "openness" is not simply a measure of software's licensing. That's not even necessarily the most important consideration, as Tim O'Reilly reminds us.

But the government's policy doesn't look beyond whether the software in question is licensed under an OSI-approved license. This is what we thought of open source five years ago, but these days, this line of thinking is increasingly outdated.

An OSI license is a fruitful beginning to an open-source policy, but if it's the end, then the Dutch government's policy begins and ends with lawyers, who are almost certainly not the best equipped to evaluate IT solutions.

Indeed, the commentary I heard today confirmed that inbound software is first reviewed by the Dutch government's lawyers. If there's not an OSI-approved license attached to it, even if the software is provided by an open-source vendor with full rights to view and modify the software (but not redistribute it), it's out.

This wouldn't be so bad if there was a plethora of alternatives in each given product category for the government to choose. But there isn't.

Hence, more often than not the government ends up buying an established proprietary solution. It's very difficult for most products to run the legal gauntlet that the government has established. The vendors that do are either too small to effectively service the government's requirements, or they're Red Hat, which focuses on a limited infrastructure product portfolio.

Having painted itself into a legal corner, there's one easy thing for the government to do: buy the same proprietary software it always has.

Given that the policy allows for selection of a proprietary product if a suitable open-source alternative doesn't exist, the stated preference for open-source solutions is turning into a minor speed bump on the way to continued acquisition of proprietary software.

This is silly.

The Dutch government should focus on the end: open, interoperable solutions. True, doing so requires more thought than a binary decision based on a license. But it's a much smarter policy to balance a range of factors (freedoms and constraints of the license, community associated with the product, open standards, payment model [license fee vs. subscription], etc.), in order to reach a more thoughtful position on a given piece of software.

Such a policy would result in more open-source software adoption, not less. It would let open-source software compete on broader criteria than the license. Open source, and the trends it has inspired, are much more than a license. Other considerations, such as open data policies, take precedence in our networked age.

The Dutch have the right intentions. But the way they're managing their open-source policy is not helping them most effectively reach the goals they seek.

In the battle of the open-source mobile platforms, developers have at least two choices: Google Android, which is open source but (relatively) closed development, or Symbian, which is open source...once it gets around to releasing the full source code.

Guess which one is winning?

Gartner expects Android to become the second-most popular mobile platform within the next few years as it continues to gobble up Symbian's declining market share.

But why?

Symbian has been dismissive of Google Android, as well as smaller upstarts like the LiMo Foundation, arguing that the latter is overly focused on middleware for wireless operators and the former is fake open source with more hype than substance.

All of which might be true, but the reality is that it seems to be working for Android. Google has been signing new handset manufacturers at a frenetic pace, while Symbian has been holding steady with Nokia...and that's about it.

Despite Symbian announcing new handsets, Google is actually shipping Android. There's a big difference between marketing and reality. Google Android offers the latter.

For all the buzz that Android gets from developers, its success owes more to handset manufacturers than to open-source developers. Handset manufacturers and wireless carriers are hungry for alternatives to surging Apple and declining Microsoft. And while others may not be seeing source code in copious amounts, handset manufacturers are apparently getting their fill.

More than this, though, Google gives them a safe, consumer-friendly brand. Symbian does not.

This is the reason Google Android is winning. It's not about developers--at least, not yet. Neither Symbian nor Android really offers developers open communities and open code.

No, the difference today is brand. Google has it. Symbian does not, and that's despite decade-long dominance of the mobile market.

Symbian still has a ways to go. It has a weak user interface (UI) that is supposed to get better, but that describes much that is wrong with Symbian today. Everything (source code, revamped UI, and resumption of market dominance) is always spoken of in the future tense.

Meanwhile, Google Android rolls on--not because it out open-sources Symbian, but rather because it out-executes it.

It's a convenient fiction that Microsoft is the source of all evil in the technology world, particularly for a vocal minority within the open-source community.

For such people, Microsoft hate is an excuse for a distinct lack of introspection, and credits Microsoft with far better execution and strategy than it actually possesses.

Microsoft CEO Steve Ballmer has a goofy laugh. I'm not sure it's an evil one.

I mention Microsoft because some within the open-source community quickly pounced on the company's inadvertent violation of the GPL in its Windows 7 USB/DVD Download Tool. Microsoft's Peter Galli was quick to acknowledge it:

[The license violation] was not intentional on our part. While we had contracted with a third party to create the tool, we share responsibility as we did not catch it as part of our code review process.

As conspiracies against open source go, it sounds pretty harmless--because it probably is. Open-source licensing is complex enough and the process for acquiring open-source software is loose enough, that there is room for all sorts of error, both nefarious and benign.

Guess what? People--and corporations filled with people--make mistakes. Even Microsoft. If it was as evil as some suspect, the devil himself would be out of a job.

As open-source adoption dramatically increases, we should expect to see errors of this kind increase, and not out of any sinister plan to pilfer open-source code. Errors are natural and are evidence that adoption is spreading beyond the inner sanctum of open sourcerors.

We shouldn't expect open-source adoption to be flawless or painless.

Consider Symbian. The foundation decided to aggressively embrace open source as a way to guide it to an optimistic future, but the process of open-sourcing its code is taking time. A lot of time. As Rich Sands suggests, Symbian may actually be taking too much time, frustrating its community and allowing Google Android to assume the leadership position in open-source mobile platforms.

Who knew that giving away things for free could be so hard?

It's tempting to think that open source should be an automatic reflex for companies and individuals alike. It's not. It takes time to learn how to do it properly, and even then mistakes are possible. Perhaps likely.

In the case of its Windows 7 tool, Microsoft screwed up. It's not the first time, and it's not the last.

But error is not evil.

There's something different about the Apache Software Foundation. While Apache hosts some of the world's most important software development, its members seem more concerned with good code than good politics.

It's no secret that I've become enamored lately with the Apache License, but it's less well-known what first attracted me to the license: the wonderfully nice people affiliated with Apache. From Greg Stein to Geir Magnusson to Brian Behlendorf, it's hard to find a jerk at Apache. I'm sure they exist, but they hide pretty well.

In fact, in a presentation today I attended at SAP in Walldorf, Germany, Apache Software Foundation President Justin Erenkrantz called out the importance of good manners to good governance at Apache:

There are going to be people on an open mailing list who are idiots, or maybe they're just having a bad day. Don't feed the trolls. Don't become a poisonous person.

It seems like reasonable advice, but it's discouraging to see this basic rule of polite society regularly broken within the wider open-source community. Some feel that a license to code is a license to shout others down. It's not. At least, not at Apache.

Perhaps this is particularly important to Apache because of the way it manages project development. It's one thing to be open source but, as I've written recently and as Erenkrantz highlighted in his presentation, open source doesn't necessarily equate to real openness:

You see a lot of people doing open source, but not a lot of people doing open development...At some open-source projects [Erenkrantz mentioned Mozilla], all of the technical decisions, even if the license is open source, are not subject to public comment. At Apache, everything is done in the open over public forums.

Or, as Day Software's Roy Fielding says, "If it doesn't happen on-list, it didn't happen."

Such transparent development creates great software, given that it fosters a true meritocracy. You know exactly who's doing what at Apache: it's all on the mailing lists.

Erenkrantz also noted a few other interesting aspects of Apache:

• Each Apache project is independent, which means that status on one Apache project is not fungible to another Apache project. I can be a core committer on the Apache HTTP project and it won't get me any brownie points with the Apache Cocoon project.
• Microsoft was a sponsor before it was a contributor. Its sponsorship was meant to send a message to Microsoft internally that it was OK to contribute to Apache projects.
• Erenkrantz stressed that Apache developers tend to believe that code, not licensing, should motivate contributions. Apache doesn't believe in forcing contributions through licensing or other mechanisms.

It's a great way to do development and, as Day Software and other companies have discovered, it's also a great way to do business. Open-source development, done openly.

And no jerks allowed.