As cloud computing edges its way into the enterprise, the open-source Apache Hadoop project may well prove to be the poster child of the movement. Hadoop effectively gives enterprises the power of Google or Yahoo Web indexing for free, or for the cost of a CloudEra subscription if you want to involve Hadoop's core developers in your rollout. Credit card giant Visa is an early corporate adopter of Hadoop, and points to a bright future for the open-source project.
I caught up with Visa's Joe Cunningham, head of the technology strategy and innovation group, to talk about the company's adoption of Hadoop.
Q: What got you interested in Hadoop initially and how long have you been using Hadoop?
Joe Cunningham: It's early days for us here at VISA for Hadoop. It's still very much classified as a research and development activity.
My role is the head of technology strategy and research and development for the company. Our task is to look outside the company for interesting technologies on the landscape and identify potential opportunities for those technologies to add value to either the VISA business of VISA technology and then bring them in and play with them in our lab research environment until they are ready for mainstream or commercial activity.
Hadoop is one of those technologies we've been looking at for about a year and we think it offers certain value as an augmentation to existing systems and capabilities VISA has.
Q: How do you use Hadoop at VISA? What made you think it could be the best solution for what you're trying to accomplish?
Cunningham: The most important thing to remember is VISA obviously has a heritage of offerings--very large, very scalable, very reliable, and very secure services to the payments industry. And we're continuously trying to innovate and make those services more valuable to our clients and ultimately to cardholders.
We have a data challenge we attempt to meet every day in terms of the number of transactions we handle and therefore we think there's an opportunity to look at the skills VISA already has in the data analytics space with the power of Hadoop to handle very, very, very large volumes of data.
To put that in context, we handle approximately 200 million transactions a day at VISA. That works out to be about 8,000 transactions a second, and with that comes huge volumes of data and Hadoop offers the potential to harness some of that along with some of our existing capabilities to extract more value from those transactions.
Q: Are there particular directions in which you'd like to see Hadoop evolve?
Cunningham: I think we're interested in looking at Hadoop and looking at its evolution over time. We're certainly interested in how the Hadoop community continues to operate in this open-source environment.
My specific interest is how can Hadoop evolve from the alpha beta environment in which it is today to the mainstream and how can we continue to integrate it as a mainstream technology with all the existing platforms we have here at VISA.
I'll give you two examples. The operations management space is very important to us: how we guarantee the reliability and security of our systems and how Hadoop can be merged or integrated into that environment. And secondly, and I guess this is a common question, but how can we enable SQL-like access to some of the data via the Hadoop file system or via the Hadoop engine?
Q: Given that it's still early days for Hadoop at Visa, it's interesting that you're speaking at the upcoming Hadoop World conference, along with JP Morgan Chase, China Mobile, and other Hadoop users that may be further along the adoption curve. What are you going to be talking about?
Cunningham: I plan to talk about the application stream, so I'll be taking a business-focused view of how we see Hadoop offering value to Visa. If there are tech junkies in the room, they are probably not going to be as interested in what I talk about.
I plan to spend a little bit of time showcasing Visa's technology today to set the scene. I will talk a little bit about our research and development function and how it works with the rest of Visa. Then I'll spend some time expanding on what I call our information products business.
The information products business for Visa offers services to our clients that are, obviously, information-based. So, some of the use cases where we see Hadoop potentially offering value in the future are in the areas of transaction analysis (particularly for risk products and the modeling of risk scenarios), fraud analysis (assisting our clients in potentially managing fraud more carefully), and in the loyalty space where Visa offers services on behalf of our clients to cardholders.
There's an opportunity for us to combine the power of Hadoop with data analytics capabilities that Visa has to augment those services and products on behalf of our clients.
In fact, that's an area that I'm hoping to learn a lot at the event. In some industries, Hadoop is very much mainstream but for some others, it's still emerging and I'm trying to understand whereabouts on that hockey stick or [Gartner] Hype Cycle Hadoop is, or whether Hadoop is already mainstream and it's just a matter of us catching up.
It's always good to gauge and plot that evolution. I think you need to get to these events and talk to other companies and key leaders in the community to really understand where we fit and what we should be doing next at Visa.
For those interested in attending Hadoop World in New York, the organizers are giving Open Road readers a 25 percent discount if you register by September 24.
Richard Stallman, godfather of the free-software movement and co-author of the GNU General Public License (GPL), has apparently found a home:
(Credit:
Steve McLoughlin (Alfresco))
It's not clear whether it's a retirement home, a private club for members, or what, but I'm sure they'll welcome him anytime he's in London, where it's located. Let's hope it's nearby Microsoft CEO Steve Ballmer's Zune House, as the man doesn't seem to have much love for the iPhone, according to this story in Ars Technica.
(I think GPL, in this case, actually stands for Garden Picture Library, but...don't tell Stallman. :-)
Microsoft has had some exceptional people driving its open-source strategy over the years.
Now, with Sam Ramji, senior director of platform strategy and Microsoft's point man on open source, officially leaving for a Silicon Valley-based start-up at month's end, Microsoft has the opportunity to select someone who will ramp its open-source engagement to the next level.
Should Microsoft choose a pragmatist or an anarchist?
Sam Ramji
It's a provocative question, but one that becomes easier to answer when you consider the state of open source at Microsoft and how its various open-source leaders have managed it.
Initially, it was Jason Matusow who took the bullets for Microsoft (and protests/pickets) as it cut its teeth on open-source engagement. Later, Bill Hilf took the reins and moved Microsoft's open-source involvement from discursive engagement to practical deployment: the company opened its open-source interoperability lab, and Hilf lobbied to partner with a variety of open-source companies.
Then came Ramji, who brought open-source credibility to his role at Microsoft, having used it extensively at a previous start-up. Ramji helped to kick off CodePlex, Microsoft's open-source code hosting site and has actively educated Microsoft on open source as much as he has worked with the open-source community to understand Microsoft. (Ramji is also becoming interim president of Microsoft's newly created CodePlex Foundation.)
Microsoft, as Hilf explained Thursday in announcing Ramji's departure, is far more advanced in its open-source activities than it once was, which suggests the ideal replacement for Ramji:
The perspectives on OSS at Microsoft have evolved to the point where Microsoft's open source strategy is no longer just locked in a single 'lab' on campus - now OSS is an important part of many product groups and strategies across the company. We have become increasingly clear on where we work with open source - development methodologies, projects, partners, products and communities - and where our products compete with commercial open source companies or platforms. Today, there are engineering and business leaders across the company, myself included, looking at how to drive interoperability for customers and as a lever for new growth.
Open source is, in other words, increasingly part of the standard fabric of Microsoft's technology and business strategy. As such, it doesn't need a missionary so much as a diplomatic, pragmatic messenger to discover areas within the company where open source can take greater root and to engage with the community outside Redmond.
Microsoft doesn't need a talking head, someone to fill panels at every open-source conference and pontificate on the immaculate conception of open-source code. Rather, it needs someone to help motivate Microsoft's rank-and-file to get involved in such events and to intelligently explain Microsoft's diverse and sometimes seemingly contradictory positions on open source--a fact that shouldn't be surprising to anyone who has worked at a big company.
Microsoft doesn't need a Che Gates, someone who believes open source is The One True Way and is afflicted with the unhealthy and unhelpful Microsoft-hating disease. Such a person will never be heard within Redmond and rightly so: the world has already figured out that open source is a powerful means to develop and distribute software, but it's not the cure for global hunger.
Rather, Microsoft needs a thoughtful mediator to deepen its engagement with the wider open-source community while continually reminding its employees to consider open source in product and business decisions.
In short, Microsoft needs someone who can credibly advocate for open source without being consumed by mindless rhetoric. Someone, in other words, very like Matusow, Hilf, and Ramji, but probably with a shorter Microsoft tenure (similar to Ramji). Any ideas? Send them to Hilf.
To end on a personal note...Sam, you have been incredibly generous to me, usually when I least deserved it. You have been patient and forbearing. I think the world of you. Your new start-up is lucky to have you. The only area in which you failed is I can't remember a single Arsenall ticket being sent my way. But we all have failings. :-)
Follow me on Twitter @mjasay.
Most vendors must guess what customers want to buy, and how they'll use it. For IBM, however, with about 400,000 employees, it has the potential to be its own best laboratory, one that becomes even more potent when mixed with active participation in open-source communities.
That potential, as I discovered in an interview on Friday with Jeff Schick, IBM's vice president of social software, isn't a "gimme," but is powerful if you can enable the right sort of corporate culture and processes.
For example, Schick mentioned that IBM has a technology adoption program for employees that spans the gamut of new products, add-ons and patches to existing products, and still-raw technologies direct from IBM's labs. While the invitation list and process is different for each particular item, IBM generally encourages its product groups to "experiment" upon each other. The earlier in the development process, the better.
At the heart of this open approach to technology adoption are open standards and open source. When I pressed Schick on the relative importance of both ("If you could only choose open standards or open source, which would it be?"), he responded:
Our products may include open-source components, and often do, but ultimately open standards are the most important consideration for customers. As customers integrate our products into their various enterprise systems, open standards are critical for ensuring they work.
Point taken, but it's impressive just how much open source influences IBM's product development. Gartner estimates that 80 percent of commercial applications will include open-source components by 2012. At IBM, the number may even be higher.
Despite IBM not releasing its core software products under open-source licenses, Schick noted just how integral open source is to IBM:
From a development perspective, as we build our social software products in Lotus, we're always looking at ways to improve quality and time-to-market. Open source often helps us with both areas.
For example, we were blogging within IBM for a long time before deciding to build the Lotus Connections product, which is fast approaching hundreds of millions of users. After some study, we decided to build the blogging piece of Lotus Connections using the Apache Roller project, an open-source Java blog software. We have become active contributors to the project since then.
But it's not just in Lotus Connections. As you look across nearly every capability across our social-software strategy, open source plays a critical role. Open source is an integral part of how we build products. Our engineers are very much in tune with the wide variety of open-source components that are available to them, and use and contribute to them. Regularly.
IBM seems to have figured out better than most how to marry the global open-source laboratory with a massive internal laboratory. Talking to Schick, there appears to be a very blurry line between "internal" development and "external" development, giving the company a significant advantage over proprietary (Microsoft) and open-source (Liferay, Open-Xchange) competitors alike.
Some competitors may be able to match IBM's scale, but few to none have managed to marry internal scale (employees) with the power of external scale (open-source communities) in the way that IBM has.
Follow me on Twitter @mjasay.
Peter Fenton
No other investor has had as much success in open-source software as Peter Fenton, general partner at Benchmark Capital.
A competitive triathlete, Fenton has turned the standard marathon of open-source business-building into a sprint, churning out four big open-source sales--JBoss ($350 million), Zimbra ($350 million), XenSource ($500 million), and SpringSource ($420 million)--while most investors have yet to turn a profit on any.
Not that Fenton is a one-trick pony. He also just sold FriendFeed to Facebook and sits on the board of Twitter. It's fair to say that Fenton can now afford a second Aston Martin.
But Fenton is still busy, sitting on the boards of open-source companies Pentaho, Terracotta, and Engine Yard. He's also willing to share the secrets to his open-source success, telling The Wall Street Journal the key to building a winning open-source business.
Spoiler? Build a direct line to your customers using open source and then ensure an excellent product to pave the way to adoption, then usage, then sales. According to the Journal article:
Rather than "expensive sales efforts and negotiations with the upper management to get the most money possible," the people that will be using the software can easily download and try the product. This helps the best products proliferate and weeds out the underperformers.
"If you don't have the best product, you're not going to make it in open-source," unlike traditional enterprise software, where customers often flock to good-enough products.
Having a well-received product not only results in plenty of downloads, users and developers, it also makes the sales process that much easier. With SpringSource, "anyone the company sold to was already using the product," he said.
Sounds easy, right? Well, no, not if you've ever been involved in an open-source business. Building a great product is hard enough, but doing so in a transparent fashion while encouraging active adoption without appearing faux to your community...? That's hard.
Venture investing may be more art than science to some, but Fenton has done more than most to turn open-source investing into a science, as VentureBeat reports. For instance, many open-source companies are ecstatic to have widespread adoption, but Fenton is careful to call out the difference between adoption and actual usage, as he does in this Benchmark presentation (PDF).
In this presentation Fenton calls out two strategies for investing in either "farm-raised" or "free-range" businesses. Think of these categories as company-led (e.g., Zimbra) or community-led (e.g., SpringSource) open-source businesses. Neither is better than the other: they simply refer to whether an open-source community predates a company set up to monetize it.
The strategies Fenton takes depends. For "free range," it looks like this:
(Credit:
Peter Fenton (Benchmark Capital))
For "farm raised," Fenton's strategy looks like this:
(Credit:
Peter Fenton (Benchmark Capital))
All of which means your next open-source investment or company should be a snap, right? Maybe not. It's one thing to call the correct shots--and quite another to make them. Part of the reason Fenton has been so successful is that he has invested in exceptional operators at each company, including Marc Fleury and Rob Bearden (JBoss); Satish Dharmaraj, Scott Dietzen, Andy Pflaum, and John Robb (Zimbra); and Rod Johnson and Rob Bearden (SpringSource), among others.
Perhaps this is really the key to Fenton's success, after all is said and done: he knows how to attract top-tier entrepreneurs to top-tier open-source communities. That's not something one accomplishes with a jog or casual bike ride. That's the work of a triathlete, which makes Fenton perfect for the job.
Follow me on Twitter @mjasay.
Bob Bickel, often credited with being one of the driving forces behind JBoss' success, has been in semi-retirement since Red Hat purchased JBoss, but he's apparently sticking his head out of hibernation to launch a new venture.
Look, Ma! I'm starting a new company.
If this doesn't sound very open source-y of Bickel, it's because it's not. But it does reflect Bickel's shifting interests: some might remember that Bickel helped to start Ringside Networks after the JBoss acquisition, an open-source competitor to Ning, the social-network platform company.
Not that Bickel is completely abandoning open source. He writes that "our site will have focus on running races, it will be simple to use, and it will be much more cost effective than the alternatives in the marketplace." I'm guessing "cost-effective" means he'll be using open-source technology at the heart of the Web site.
But then, that's an easy guess: who isn't using open source to power the Web these days?
Follow me on Twitter @mjasay.
Roger Shuttleworth?
Turning to the sports news, I understood why:
Shuttleworth, masquerading as Roger Federer, had just beaten Andy Roddick to win Wimbledon and beat his own record for most Grand Slam singles titles in men's tennis history.
Not bad for a computer geek.
Shuttleworth was, of course, the first African in space, but it's nice to see him getting some exercise and devoting himself to terrestrial pursuits from time to time.
Of course, it's possible that the whole Ubuntu thing is actually Federer's claim to fame. Perhaps Federer is the real open-source geek-cum-tennis pro while Shuttleworth is the sham?
Mark Federer?
Neither have I. Whenever Federer and I hang out, Shuttleworth is always curiously absent, yet shows up soon after Federer leaves.
Some people think Roger Federer and Mark Shuttleworth simply look like each other. But I'm not fooled. I know the truth.
They're one and the same.
Or maybe they're twin brothers separated at birth.
;-)
Follow me on Twitter @mjasay. But notice the timestamps: you'll never find me Twittering at the same time as Richard Stallman. Coincidence?
It was bound to happen. With the U.S. government promising truckloads of cash to overhaul the U.S. health care system, while simultaneously making positive noises around open source, it was just a matter of time before someone connected the dots.
That someone appears to be Joanne Rohde, former executive vice president of worldwide operations at Red Hat, who has launched the Axial Project, a stealth-mode start-up that aims to "combin[e] the principles of Open Standards and Open Source...to connect all the parties in the Health ecosystem safely and securely."
It's a big task, but then, that's precisely what open source is good for tackling.
Indeed, as I've written before, the U.S. health care system, with its myriad of providers, insurers, etc. is ripe for open source. Open source isn't a panacea, but it has proved itself adept at resolving precisely this sort of complexity, with Linux and the various Apache projects as just two examples.
I've been talking with Rohde for at least a year now--most recently meeting for breakfast in Raleigh in April--and have enjoyed seeing her ideas germinate and flower. The company has gone through various guises (and names: as late as April, Rohde was calling the company EHRmail), and is now growing to meet the challenges ahead of it.
Axial has been quietly assembling a team of seasoned veterans from Rohde's Red Hat and UBS past, including Michael Yuan and John Casey, but most recently Matt Mattox, Red Hat's director of ISV alliances, who announced via e-mail his move to Axial:
(Credit:
Matt Asay)
Axial has not yet raised venture funding, but planned to raise its seed money through alternative avenues, at least as of my April conversation with Rohde. Given the company's mission--to build an integration tool kit around a message broker for health IT companies, universities, and corporations that allows sending and receiving of data across existing infrastructures--coupled with its open-source approach and roster of seasoned executives, I'm guessing funding won't be an issue.
The real issue is whether even open source is powerful enough to fix the U.S. health care system. Good luck to Mattox, Rohde, and the Axial Project team as you seek to answer that question in the affirmative.
Follow me on Twitter @mjasay.
I've suggested before that Index Ventures could well be the "best venture firm in Europe." Index has one of the most interesting investment portfolios of any venture firm on the planet, having invested in companies like Skype, Openads, Oanda, DimDim, and others.
Mike Volpi, Index Ventures new Partner
Today, Index became even more interesting, adding Michelangelo ("Mike") Volpi to its investment team as a partner based in London. Volpi was most recently the CEO of online video company Joost, but made his name as the mergers and acquisitions maestro at Cisco, where he oversaw 75 acquisitions.
Given Volpi's background as a deal-maker, the question is whether he's going to be investing for Index or selling for Index.
The answer is "both," of course, and Volpi's recent failure with Joost should prove useful instruction for his future Index investments, as suggested in an interview with D: All Things Digital. Look for Volpi to take a hard-headed view on Internet business models: less advertising, and more transactional business models.
Index was already a leading light in the global venture investment constellation. Adding Volpi should make it even better.
Follow me on Twitter @mjasay.
I wake up at 5:30 AM to write this blog, then spend the rest of the day (and sometimes evening) working my day job at Alfresco, an open-source applications company.
Meanwhile, my brother-in-law, Josh Robbins, taunts me by sending pictures of him chumming around with Thierry Henry (FC Barcelona), Ryan Babel (Liverpool FC), Alessandro del Piero (Juventus), Mathieu Flamini (AC Milan), Steve Nash (Phoenix Suns), Tony Parker (San Antonio Spurs), and others at a Steve Nash Foundation event. His job? He's director of finance at Calle, a cool street soccer sporting goods manufacturer that he helped found.
Yes, I did get to meet Cesc Fabregas on a trip to London a few years back, but I'm still wishing that I got to dress up sports stars in cool sports gear rather than shelling out $100 to $600 to watch these same stars kick balls around whenever I go to London.
I believe in divine retribution, and I'm earnestly praying for Josh to get some. Soon.
Or maybe I should simply stop writing about low-cost, open-source software and focus on the bling? Microsoft has an executive box at the Emirates Stadium where Arsenal plays. I got to sit in it once for the opening match of the 2007/08 season. Perhaps if I agree to say something nice about Microsoft once every few weeks, they'll invite me back.
Or not.
Josh Robbins dresses Thierry Henry in Calle gear
(Credit: Josh Robbins)Follow me on Twitter @mjasay.
