Microsoft is in significant disarray, fettered by its destkop dominance as the world goes mobile. Would this have happened anyway, or is Microsoft CEO Steve Ballmer to blame?
Developers! Developers. Developers? Developers!?!?
Ballmer, after all, knows how to sing to developers, but he doesn't really speak their language. Former Microsoft CEO and co-founder Bill Gates did. Now, more than ever, Microsoft needs to get in front of developers but finds itself playing catch-up.
Gates announced his resignation back in 2006 and formally discarded his full-time Microsoft duties in 2008. But it has been a long time since Gates' hand was full time on the steering wheel.
That's a problem for the world's largest software company. It was Gates who saw the threat (and opportunity) the Internet posed for Microsoft--drafting his excellent "The Internet Tidal Wave" (PDF) memo in 1995--and alerting his troops to an array of threats that saved Microsoft from ruin...while helping it to ruin many others on its path to billions in profits.
Gates oversaw Microsoft's early, largely successful forays onto the Web. Ballmer has shepherded Microsoft to vanishing mobile market share (now just 7.9 percent of the market), a hesitant tiptoe into software as a service, and a general sense of retreat in emerging markets.
Hence, while former Microsoftie Don Dodge talks up his new employer, Google, with its food perks and 401(k), it's really the company's vision that has him jazzed:
Google has made three big bets on the future of computing; Chrome (browser), Google Apps (cloud), and Android (mobile). The trends are pretty clear. All the exciting new applications are running in the browser, with application code in the cloud, and the cell phone as the platform....2010 will be the year that enterprises of all sizes start their transition to Gmail and Google Apps, and take their first steps towards the vision of the future.
Dodge couldn't sell this sort of vision at Microsoft.
Microsoft has been playing catch-up for many years, but at least did so successfully under Gates. With Ballmer, there's a sense that Microsoft is always a half-decade too late on critical initiatives like search, open source, and mobile.
So is the problem Ballmer, or is Microsoft simply doomed, blinded by its own success with personal computers--a blindness that no CEO could overcome?
I hate to ascribe so much importance to any one person, but just as Steve Jobs is the soul of Apple and its revolutionary leader, so, too, was Gates the heart and mind of Microsoft. He understood developers, and they rewarded his belief in them by making Microsoft the world's largest software company.
Microsoft is the poorer for Gates' departure.
Even as I type this, Google keeps moving into the future while gouging Microsoft's past. TechCrunch is reporting that Google is acquiring DocVerse, which enables people to collaborate on Microsoft's Office documents. Microsoft is under siege.
This is just the beginning.
Developers are coding for Google projects, Twitter, and other new-style Web applications. Morgan Stanley is predicting the mobile market will be twice the size of the "desktop" market. Will Google someday dwarf Microsoft in size and influence?
Unless Ballmer can discover his recessive developer gene, the answer my well be yes.
Update at 2:10 AM Pacific on Tuesday: Newsweek predicts the ouster of Ballmer in 2010, but ZDNet's Mary Jo Foley cautions "not so fast."
Follow me on Twitter @mjasay.
It's reasonably clear that open source is the heart of cloud computing, with open-source components adding up to equal cloud services like Amazon Web Services. What's not yet clear is how much the cloud will wear that open source on its sleeve, as it were.
Eucalyptus, an open-source platform that implements "infrastructure as a service" (IaaS) style cloud computing, aims to take open source front and center in the cloud-computing craze. The project, founded by academics at the University of California at Santa Barbara, is now a Benchmark-funded company with an ambitious goal: become the universal cloud platform that everyone from Amazon to Microsoft to Red Hat to VMware ties into.
Or, rather, that customers stitch together their various cloud assets within Eucalyptus.
I caught up with Eucalyptus founder and Chief Technology Officer Rich Wolski to learn more about how Eucalyptus hopes to displace industry leaders like VMware, and the role that open source plays in growing the cloud-computing market.
While Wolski made a big deal about the company's open-source credentials, I found his argument about the open architecture of Eucalyptus more compelling:
... Read more[Eucalyptus] is architected to be compatible with such a wide variety of commonly installed data center technologies, [and hence] provides an easy and low-risk way of building private (i.e. on-premise or internal) clouds...
Thus data center operators choosing Eucalyptus are assured of compatibility with the emerging application development and operational cloud ecosystem while attaining the security and IT investment amortization levels they desire without the "fear" of being locked into a single public cloud platform.
Whether you're an enterprise or a consumer, ultimately your big concern in buying technology is "Will it do what I want it to do?" Sometimes components matter, but most people most of the time just want something that works.
Open source inside, but do you care?
I'm going to assume that at some point over the last 20 years you bought a car. So, how important was the car maker's use of just-in-time manufacturing to your purchase decision? I'm going to go out on a limb here and say it was of no consideration at all.
Well, I think we're fast approaching the point where open source to software will be like JIT to automotive manufacturing. While it will critical to the producers of software, woven into the fabric of its operations, it will be of no importance at the point of consumption.
As hard as this might be to accept, open source is not a value proposition in its own right.
As hinted above, this is mostly true. Customers do care about the things that open source offers (lower cost, higher quality, etc.). But they probably don't recognize (or care) that these benefits stem from open source, per se.
Consider the Web. Open-source software provides the fundamental building blocks for nearly all Web services like Facebook, Amazon, etc., not to mention the infrastructure for public and private clouds.
End customers aren't asking for open source on the Web or in the cloud. They're asking for well-managed services that solve their problems. It's the vendors who care, because it allows them to grow highly scalable businesses at a modest cost.
Indeed, many of the customer benefits of open source (i.e., the ability to view, modify, and redistribute code) disappear or get muted on the Web and in the cloud. This hasn't stopped customers from buying into the Web/cloud.
Red Hat may disagree. It's apparently betting that customers care about components in the cloud. Why else would Red Hat Enterprise Linux pricing be much more expensive than Windows on Amazon EC2, despite being much cheaper for on-premises deployments, as IBM's Savio Rodrigues finds?
But I suspect Red Hat will need to change its pricing, as the OS is even more commoditized in the cloud than it has been for on-premises deployments. Both Red Hat and Microsoft will race to the bottom in pricing, with the emphasis being on the applications that run on them.
After all, this is the thing that drives customers' purchasing decisions. Red Hat knows this, which is why it (rightly) makes a big deal of the huge ecosystem of applications that run on RHEL.
In the cloud, even more attention is focused on service that customers use. Open source, in such a world, is essential infrastructure, but it's infrastructure that every vendor shares or will soon share, making the battle all about end-user facing applications, and not about developer-facing open-source components.
This is a very good thing. It means we can get back to focusing on solving customer problems, rather than fetishing and battling over open-source licenses. It's about time.
For those entrepreneurs looking to make a living from open-source software, Index Ventures general partner Bernard Dallé has some advice: get thee to a cloud strategy.
Bernard Dallé
(Credit: Index Ventures)Why? At a time when enterprises may be less willing to spend on software, they're increasingly interested in spending on the operation of that software through cloud computing, an interest that can be bought...and sold.
The cloud isn't simply a clever way to provide social-networking services, either. As Dallé suggested in a phone interview on Wednesday, cloud computing may well be the best way to monetize enterprise-facing open-source software.
He should know. Index Ventures has been one of the most successful investors in the changing world of software, hitting home runs with MySQL, Skype, and more. So when Dallé says that as much as 70 percent of the investment opportunities they see now are cloud-related, and that this bodes well for open source, it's worth paying attention.
Given that the cloud renders software less visible to end users, I asked Dallé if cloud computing spells the end for open-source businesses. Far from it, he said:
I think it's good news. I don't think open source is going away. It's here to stay. The world is increasingly moving to a hybrid world: a combination of on-premises and cloud computing. We're not going to see a 100 percent cloud world.
If I look at our portfolio, even our "open-source companies" like Pentaho, OpenX, and DimDim are turning to the cloud to monetize their open-source software assets.
Open source provides a convenient on-ramp and off-ramp for customers, helping them evaluate the software at low to no cost and also gives a free (as in cost and as in freedom) exit in case things go wrong. Between that entrance and exit is a ripe opportunity to make a lot of money by delivering value to customers.
Dallé further explained that open source helps vendors reach customers through low-cost distribution, but cloud computing, importantly, makes the open-source software palatable to a class of customer that finds open source too risky, yet has no problem using it when hosted.
If this sounds like a potent mix, it's because it is. It's also a highly efficient, low-cost way to start and build a company. Dallé elaborates:
The other big trend, not related to open source, is cloud-on-cloud: cloud services running on other clouds. It used to be that everyone ran their own data center, but now an increasing number of companies are happily running their services on Amazon EC2 or other public clouds. This dramatically lowers the cost of starting a service, and starting a company around it.
This might raise the concern that we'll see too many open source/cloud companies, not too few. Dallé isn't worried: "The quality of an investment always comes down to the quality of the people involved and their execution."
If Dallé's correct, the right place to look for open-source businesses to flourish is at the nexus of on-premises open-source software and cloud computing. It could prove to be a potent mix. And while the cloud might not be the right delivery platform for some software, it probably does have a high degree of salience for many.
A recent survey suggests that CIOs are loosening the purse strings on IT spending. IT vendors may want to hold off their celebrations, though, because much of the spending appears to be headed for deflationary forces like cloud computing, virtualization, and their kissing cousin, open source.
An economic rebound never looked so dire.
That's unless you're an IT buyer, of course, suggests a new report from Goldman Sachs. In this week's report, titled "A Paradigm Shift for IT: The Cloud," Goldman Sachs said it expects that pent-up IT dollars will flow in the short term to building out next-generation data centers (e.g., cloud computing). But in the long term, less money is expected to find its way into fewer wallets:
After the initial build-out, Cloud Computing could drive some headwinds for the IT industry, as a result of two factors. First, we see virtualization as a deflationary technology. Second, we see IT spending consolidating in the hands of fewer buyers--the Cloud providers, hosting vendors, and large enterprises. These factors will likely dampen IT spending growth due to greater utilization and buyer pricing power.
Even short-term build-outs may prove disappointing, however, as Goldman Sachs expects large enterprises to grow existing virtualization and automation technology adoption in the rollout of private clouds, shifting slowly to an embrace of public clouds over time. The chart below gives some idea as to when cloud computing will hit its stride:
Who wins in this scenario?
According to the report, Red Hat stands to benefit from the cloud-computing craze. ("Red Hat is well positioned for the emerging Cloud Computing ecosystem, largely due to its open source background and current ubiquitous deployments in data centers, including enterprises, as well as in Cloud providers such as Amazon," the report states.)
But the real beneficiaries will be...the same old crew. "[K]ey suppliers for internal Clouds are likely to be those that have the most complete portfolio of hardware, software, and services," including IBM, Hewlett-Packard, Cisco Systems, EMC, and Oracle.
New boss...same as the old boss.
The other beneficiaries are the start-ups that provide critical components of cloud computing, with an emphasis on management tools. Here we may see open-source companies benefit, including Reductive Labs (Puppet project), Cloudera, and the two rising private cloud companies, VMOps and Eucalyptus, among others.
While open source doesn't factor heavily into this particular Goldman Sachs analysis, the firm has before called out open source's role in wringing more value out of fewer IT dollars. Open source is a primary driver of the global reset in IT spending expectations.
With less money flowing into the pockets of fewer vendors, we can expect to see both increased consolidation and fierce competition for the IT spending that remains. Those vendors that can help CIOs do more with less stand to benefit from this shift to low-cost, high-value computing.
And those that can't? Well, let's just say they may pine for the good old days of the global recession.
Ars Technica's Ryan Paul wants to know, "Can a [truly open smartphone] be done?" But the real question is, "Should we care?"
Hello? Can I get some freedom around here?
Hence, Bradley Kuhn of the Software Freedom Law Center expresses anxiety about the future of freedom in mobile...
We are in a very precarious time with regard to the freedom of mobile devices. We currently have no truly Free Software operating system that does the job.
...when he really should be concerned with choice in mobile. Right now, we're spoiled for choice in mobile, what with Apple's iPhone, Google Android, Symbian, LiMo, Moblin, etc., which suggests that users are free to move between devices.
In this case, it's not the license that makes users free. It's the market.
Open-source software plays an important role in ensuring user choice, but it's not the sum total of the freedom/choice equation. It's just one factor. As Tim O'Reilly reminds us, it's not even necessarily the most important factor, either.
Kuhn and other free-software advocates worry that the nuts and bolts making up the software on mobile phones be free, but this is surprising given the increasing irrelevance of single-node freedom when it's tied into a network. This is what I've described as "the Hotel California of tech," and it suggests we should be far more concerned with freedom between nodes than freedom of the nodes themselves.
In other words, the real concern should be over open data, not open phones. No matter how open my phone's software may be, it's meaningless if I can't move my data between devices or wireless providers.
Even here, there's cause for hope. For example, Funambol's open-source mobile cloud synchronization and push e-mail software is in use by 10 of the leading mobile service providers, as identified in a new report, which arguably should be more relevant to the Freedom fighters than whether Bluetooth is open source.
Glyn Moody, a journalist with strong free-software leanings, understands this. That's why he makes the case for an open cloud, and not simply "open node in the cloud":
Ideally, what we need is a completely open source cloud computing infrastructure on which applications providing people with things like (doubly) free email and word processing services could be offered....The trick here is not to fight the battle on the opponents' terms, but to come up with something completely different.
For example, how about creating an open source, *distributed* cloud? By downloading and running some free code on your computer, you could contribute processing power and disc space that collectively creates a global, distributed cloud computing system. You would benefit by being able to use services that run on it, and at the same time you would help to sustain the entire open source cloud ecosystem in a scalable fashion.
One can quibble with the feasibility of this approach, but at least Moody is thinking at the right scale. Those who are still stuck in the Open Source 1.0 of isolated, client-side software are not.
I suppose someone has to fixate on upper-case Freedom above all other priorities. Like usability. Or ubiquity. Or...well, anything.
But most of us don't think this way, because the world is a lot more complicated than Freedom on one hand, and Slavery on the other. Also, the focus of freedom has evolved in our networked world, though some free-software advocates seem mired in Freedom 1.0.
It's time to upgrade. Freedom is more than a license. It derives from a competitive market, one that is assisted by open source but not exclusively or even primarily defined by it.
Google was born on the Web and is increasingly giving Microsoft fits by forcing the decades-old software giant to compete on Google's terms. Like open source. Like cloud computing.
Microsoft may shore up its fortunes in the short term with a successful Windows 7 launch. But in the long term, its very success with outdated "desktop" products threaten to cede the market to Google.
We'll have all of it, please
It's not really fair to Microsoft. Microsoft is a victim of its own success, needing to cater to its existing clientele with each new release, in true "Innovator's Dilemma" fashion. Hence, Microsoft continues to make a lot of money, but its last two quarters have seen traditional strengths like Windows become a drag on earnings as enterprises spend more money with Google, Red Hat, and others.
Google's lack of legacy frees it to innovate rapidly and broadly, as Genentench CIO Todd Pierce, a Google Apps customer, suggests:
The rate of innovation at Google is - well I mean, the Oracle, SAP and Microsoft product cycle is five years; Google's product cycle is five days. It's incremental. In five days you're not going to be able to cancel your Microsoft Office license, but in five years, you won't have Microsoft Office.
Microsoft, for its part, is so concerned with "backward compatibility"--"Is this product/feature compatible with our ability to continue to monetize our 1980s-style desktop monopoly?"--that it continues to struggle to embrace the Web. CNET blogger Dave Rosenberg points out that Windows 7 should have been Microsoft's launchpad to cloud computing, but isn't.
There are a lot of "should have beens" for Microsoft when it comes to the Web.
Meanwhile, no one is slowing down for Microsoft. Let's stick with cloud computing for a minute. VMware dominates virtualization and has a strong claim on cloud computing, though open-source rivalry from Eucalyptus and VMops threatens to challenge both VMware and Microsoft as they seek to dominate cloud computing.
And then there's Google, which provides an increasingly wide array of cloud-based services to enterprises looking to untether themselves from the desktop. In an interview with CNET News, Google CEO Eric Schmidt argues that "The browser can be both enterprise- and consumer-capable. The architecture is driven from the browser. That is the story of enterprise IT today."
In other words, the desktop is simply the means by which a user loads a browser. It's a gateway. The value is not in the desktop anymore. It's in the browser, which is the new desktop, in terms of real functionality delivered.
Microsoft's big opportunity to stymie the threat from Google and others is SharePoint. Microsoft CEO Steve Ballmer has described it as Microsoft's new operating system, but it's in a recent interview with Forrester that he makes this meaningful:
In my own mind I compare (SharePoint) to the PC, the PC started off life as a spreadsheet machine, then became a programming machine, a word processing machine, (SharePoint is) a general purpose infrastructure that connects people to people and people to information....
I think SharePoint is considered a very serious development platform for rapid application development (by IT architects and developers).
SharePoint is Microsoft's best attempt to connect desktop applications like Office with centralized, cloud/cloud-like collaboration and storage. Yes, Microsoft has other initiatives like online Office, but none marries so well its legacy profit centers with future innovation. And, given that SharePoint is already a $1 billion and frenetically growing business, it has momentum that other initiatives don't.
SharePoint, then, may be Microsoft's best hope for marrying its legacy to the future of Web-based computing.
Microsoft needs something like this. It is losing in mobile, and not simply to Apple. Google's Android momentum is almost astounding, with AdMob data pegging Android smartphone penetration in the U.K. at 10 percent, as but one example.
If we assume that mobile will increasingly be the client platform of choice, then we see Google squeezing Microsoft from the top (cloud) and the bottom (client).
In both areas, open source is Google's weapon of choice, and it's one that Microsoft is going to have to figure out quickly if it wants to be a player on the Web. The Web is too big for Microsoft to control it, and the Web is overwhelmingly open source, as Lotus founder Mitch Kapor states:
The accomplishment of open source is that it is the back end of the Web, the invisible part, the part that you don't see as a user.
All of the servers, pretty much, they run Linux as the operating system; they run Apache as the basic Web server on top of which everything else is built. The main languages out of which Web applications are built - whether it's Perl or Python or PHP or any of the other languages - those are all open source languages. So the infrastructure of the Web is open source ... the Web as we know it is completely dependent on open source.
Kapor further suggests that Microsoft's war with open source is over, or should be over: open source has won. It's essential infrastructure now, and hence something that Microsoft needs to embrace, not fight. This isn't about open-source religion. It's about pragmatism. Pragmatism that Microsoft, like anyone else, can embrace.
Google is using the future (open source, cloud) to compete for the future, and its tactics threaten to hit Microsoft in its profit centers like Windows.
Microsoft, however, appears to be mired in its past. Windows 7 looks to be a serious upgrade over its Vista predecessor, but in 10 years time, will we care? Or will we have moved on, forgetting about those quaint days when we used to care about the operating system and applications like Office?
Follow me on Twitter @mjasay.
In the cloud, no one cares about your software license. That is one of the most liberating--and frustrating--things about cloud computing.
Depending on your perspective, it either opens up computing or closes it off. Customers don't seem to care one way or another, happily shoveling data into cloud services like Google, Facebook, and others without (yet) wondering what will happen when they want to leave.
Cloud computing may just be the Hotel California of technology.
Google Trader
(Credit: Google)I say this because even for companies, like Google, that articulate open-data policies, the cloud is still largely a one-way road into Web services, with closed data networks making it difficult to impossible to move data into competing services. Ever tried getting your Facebook data into, say, MySpace? Good luck with that. Social networks aren't very social with one other, as recently noted on the Autonomo.us mailing list.
For the freedom-inclined among us, this is cause for concern. For the capitalists, it's just like Software 1.0 all over again, with fat profits waiting to be had.
The great irony, of course, is that it's all built with open source.
In this cloud computing/Web 2.0 world, infrastructure needs to be cheap, flexible, and plentiful. Open source delivers all three.
Hence, we've seen companies like MySpace tripping all over themselves to open up parts of their platforms in order to make themselves more appealing to developers. As ReadWriteWeb wrote of Facebook back in 2007, however, such developer outreach has not opened up these Web platforms in the sense of providing useful off-ramps to services like Twitter, Digg, Facebook, etc. It has simply created more on-ramps.
Cue the nefarious Microsoft theme song.
Rather than wringing our hands over this, I think there's an opportunity to create amazing amounts of good (and wealth) in this open/closed Web. Frankly, the longer we're in this, the less it's going to matter whether the code is open or closed because, as Tim O'Reilly has been saying for years, data is the heart of the Web, and even open data isn't going to hurt a successful vendor's network effects.
Take Google Trader, an interesting new SMS application that helps people buy and sell goods through text messaging. As The Economist notes, however, one of Google Trader's most interesting applications is in helping to foster free markets in emerging economies:
Lastly there is Google Trader, a text-based system that matches buyers and sellers of agricultural produce and commodities. Sellers send a message to say where they are and what they have to offer, which will be available to potential buyers within 30km for seven days. Mr Makawa says his father used the service to look for a buyer for some pigs, which he sold to pay school fees. These services cost 110 shillings ($0.05) a time, the same as a standard text message, except for Google Trader, which costs double that. In their first five weeks the services received a total of more than 1m queries.
I'm not familiar with the economics of SMS, but I'm guessing that Google gets a cut of the messages its application generates. The more useful Google Trader becomes, the more SMS it generates, the more commissions Google collects.
For the entrepreneurs using Google's service, they could possibly care less whether Google Trader is open source, but Google might. Open the source (and the API to the service), and let a thousand add-on development projects bloom. The more useful and feature-rich the Trader application, the more SMS, the more...you get the picture.
Take me to Google, Earthling.
The key is to create an open Web platform, one into which a diverse array of mobile software services can tie. This is one reason Google is such an advocate for open source. Android and other projects bring more people to the Web, a Web that Google monetizes through proprietary services like AdWords.
The community is critical to building upon the platform, but the money is in control of the platform and provisioning of services therefrom.
Just ask Amazon.com. According to ZDNet, Amazon's Elastic Compute Cloud (EC2) service makes roughly $220 million per year. That's a lot of cash, and is a function of EC2 sitting at the heart of a growing developer community, one that builds upon Amazon's open APIs to the service.
Some companies like Cloudera and Red Hat will make piles of cash providing the infrastructure for this cloud-computing gold rush. But the biggest money of all will be those that can build platforms in the cloud, platforms that depend upon open source but which aren't open in the traditional open-source license sense of the word.
That traditional licensing world is dead. Open-source licensing has become an on-ramp to closed data services, hardly what its creators envisaged. In fact, proprietary cloud vendors are almost certainly going to become the biggest cheerleaders for open source, because it means more developers creating more on-ramps to the cloud.
Even if such providers create effective exits, it's unlikely that consumers and businesses will actively use them...
...just like in the Software 1.0 world.
Follow me on Twitter @mjasay.
The platform wars are over, and open source has won. It's not that open source has displaced Windows or the iPhone or anything else, but that every platform will necessarily include open source. It's simply too expensive and too difficult to go it alone anymore, whether you're an aspiring start-up or Microsoft.
IDC captures this thought in a recent Asia Pacific survey, which highlights open-source software as a foundation for flexible platforms, rather than as point solutions:
Vendors position [open source] as a solution, rather than a point product, by customizing to the needs of specific verticals....Other perceived benefits of adopting open source, apart from the traditional cost savings, include no vendor lock-in, access to the source code, and the flexibility to further customize the software to match individual needs. All these in turn increase the ease of integration with the existing infrastructure of an organization, as well as the compatibility with different platforms. This gives the organization an opportunity to use and test open source without changing their whole IT infrastructure.
It's this flexibility that is arguably open source's biggest benefit, and why companies like Yahoo are actively contributing to open-source projects. Yahoo's senior vice president of cloud computing, Shelton Shugar, argues,
We believe that the developer community is a key component in making Yahoo! a success. The challenges the industry is facing today in terms of large-scale, global cloud solutions are bigger than any one company (big or small) is able to solve on its own. As we contribute to the [open-source] community, we also learn from the community, and third party developers are a valuable resource helping to speed innovation.
Companies that care about developers must care about open source. Like Amazon with its Kindle. Or Microsoft, whose CEO famously sang the praises of developers. So long as Google and its crowd compete using open source, Microsoft will, of necessity, follow suit.
It's not about peace and love. It's about capitalism and competition. That's the new face of open source.
This isn't to suggest that the world will go 100 percent open source tomorrow. But we'll see a lot more open source as vendors strive to meet CIO's need to cut costs while boosting productivity, and as they seek to become flexible platforms to meet the demands of increasingly complex enterprise IT requirements.
As cloud computing edges its way into the enterprise, the open-source Apache Hadoop project may well prove to be the poster child of the movement. Hadoop effectively gives enterprises the power of Google or Yahoo Web indexing for free, or for the cost of a CloudEra subscription if you want to involve Hadoop's core developers in your rollout. Credit card giant Visa is an early corporate adopter of Hadoop, and points to a bright future for the open-source project.
I caught up with Visa's Joe Cunningham, head of the technology strategy and innovation group, to talk about the company's adoption of Hadoop.
Q: What got you interested in Hadoop initially and how long have you been using Hadoop?
Joe Cunningham: It's early days for us here at VISA for Hadoop. It's still very much classified as a research and development activity.
My role is the head of technology strategy and research and development for the company. Our task is to look outside the company for interesting technologies on the landscape and identify potential opportunities for those technologies to add value to either the VISA business of VISA technology and then bring them in and play with them in our lab research environment until they are ready for mainstream or commercial activity.
Hadoop is one of those technologies we've been looking at for about a year and we think it offers certain value as an augmentation to existing systems and capabilities VISA has.
Q: How do you use Hadoop at VISA? What made you think it could be the best solution for what you're trying to accomplish?
Cunningham: The most important thing to remember is VISA obviously has a heritage of offerings--very large, very scalable, very reliable, and very secure services to the payments industry. And we're continuously trying to innovate and make those services more valuable to our clients and ultimately to cardholders.
We have a data challenge we attempt to meet every day in terms of the number of transactions we handle and therefore we think there's an opportunity to look at the skills VISA already has in the data analytics space with the power of Hadoop to handle very, very, very large volumes of data.
To put that in context, we handle approximately 200 million transactions a day at VISA. That works out to be about 8,000 transactions a second, and with that comes huge volumes of data and Hadoop offers the potential to harness some of that along with some of our existing capabilities to extract more value from those transactions.
Q: Are there particular directions in which you'd like to see Hadoop evolve?
Cunningham: I think we're interested in looking at Hadoop and looking at its evolution over time. We're certainly interested in how the Hadoop community continues to operate in this open-source environment.
My specific interest is how can Hadoop evolve from the alpha beta environment in which it is today to the mainstream and how can we continue to integrate it as a mainstream technology with all the existing platforms we have here at VISA.
I'll give you two examples. The operations management space is very important to us: how we guarantee the reliability and security of our systems and how Hadoop can be merged or integrated into that environment. And secondly, and I guess this is a common question, but how can we enable SQL-like access to some of the data via the Hadoop file system or via the Hadoop engine?
Q: Given that it's still early days for Hadoop at Visa, it's interesting that you're speaking at the upcoming Hadoop World conference, along with JP Morgan Chase, China Mobile, and other Hadoop users that may be further along the adoption curve. What are you going to be talking about?
Cunningham: I plan to talk about the application stream, so I'll be taking a business-focused view of how we see Hadoop offering value to Visa. If there are tech junkies in the room, they are probably not going to be as interested in what I talk about.
I plan to spend a little bit of time showcasing Visa's technology today to set the scene. I will talk a little bit about our research and development function and how it works with the rest of Visa. Then I'll spend some time expanding on what I call our information products business.
The information products business for Visa offers services to our clients that are, obviously, information-based. So, some of the use cases where we see Hadoop potentially offering value in the future are in the areas of transaction analysis (particularly for risk products and the modeling of risk scenarios), fraud analysis (assisting our clients in potentially managing fraud more carefully), and in the loyalty space where Visa offers services on behalf of our clients to cardholders.
There's an opportunity for us to combine the power of Hadoop with data analytics capabilities that Visa has to augment those services and products on behalf of our clients.
In fact, that's an area that I'm hoping to learn a lot at the event. In some industries, Hadoop is very much mainstream but for some others, it's still emerging and I'm trying to understand whereabouts on that hockey stick or [Gartner] Hype Cycle Hadoop is, or whether Hadoop is already mainstream and it's just a matter of us catching up.
It's always good to gauge and plot that evolution. I think you need to get to these events and talk to other companies and key leaders in the community to really understand where we fit and what we should be doing next at Visa.
For those interested in attending Hadoop World in New York, the organizers are giving Open Road readers a 25 percent discount if you register by September 24.






