The Wisdom of Clouds

Microsoft's announcement of Windows Azure pricing confirmed a lot of speculation about the nature of Azure and its target audiences.

First, Microsoft plans to compete directly with infrastructure-as-a-service (IaaS) vendors, especially market pioneer and leader Amazon Web Services. As CNET's Ina Fried reported:

On a pure consumption basis, Microsoft said it will charge 12 cents per hour for computing, 15 cents per gigabyte for storage and 10 cents per 10,000 storage transactions. For network bandwidth, the software maker is charging between 10 cents and 15 cents per gigabyte.

Interestingly, The Burton Group noted on their blog that something is missing in Microsoft's initial pricing for consumption:

As you might expect, the compute model is similar to EC2 in that the pricing is "per hour" and per GB. The missing part in the model is the size (or type in EC2 terms) of the compute platform. I would expect Microsoft to augment pricing for compute based-on the amount of compute resources an application requires.

That is a pretty glaring omission. I wonder if it reflects the maturity of Microsoft's virtualization technology, or if the company is holding back some interesting details about how varying compute demands will be handled. I expect we'll find out when the company holds its Professional Developer's Conference in November.

Second, like Amazon, it's an IaaS offering that really verges on a platform-as-a-service (PaaS) model: targeted more at developers than traditional system administrators. The proof is in the importance placed on .NET services, and the way messaging and access control operations are priced. The Windows Azure Platform Pricing and Licensing Overview states:

• Messages = $0.15/100K message operations , including Service Bus messages and Access Control tokens
• Bandwidth = $0.10 in / $0.15 out / GB

Interestingly, that page positions these services to be about integration with existing, on-premises enterprise software systems:

Messages (Includes Access Control, Orchestration, and Reliable Queuing for message): .NET Services allow developers to easily connect their cloud applications and databases with existing software assets and users. This connection between cloud and on-premises assets is facilitated by the exchange of messages. The consumption-based pricing model means that customers will pay only for the number of message operations that their applications use. The definition of a "message operation" includes Service Bus messages and Access Control tokens. Messages are charged to the customer in discrete blocks of 100,000 ("100k") for each monthly billing period...

However, if I read this right, any request to the Azure access control system will count as a .Net operation, resulting in a charge per 100K operations. (Bandwidth charges are also charged, but I am assuming that is only so if the message is sent or received outside of the Azure service.)

Which brings me to the point of this post. Will Azure force developers to be much smarter about their use of network bandwidth and .Net services than they would be otherwise? Is this pricing a sign that we are returning to the world in which resources will be carefully rationed amongst operations, and that efficiency reigns supreme? Will companies compete on the efficiency of their developer's architectures and implementations?

Do cloud computing services like Windows Azure or Amazon Web Services encourage better software development, or will they have little effect on the quality of commercial software?

IBM, Microsoft, Cisco, Intel, the IEEE/ISTO, and key members of the Cloud Computing Interoperability Forum met recently to address how they could work with the community to drive cloud computing markets and technologies forward. Jesse Silver, one of the CCIF's four co-creators, spoke to me after the meeting, and Reuven Cohen released a single paragraph of minutes on his blog Tuesday morning:

Yesterday representatives of CCIF, CloudCamp, Cisco, IBM, Intel, Microsoft, and the IEEE-ISTO met while attending the Cloud Computing Expo in New York. Other companies were invited but were unable to attend, generally due to the short notice. The companies agreed on a shared goal to promote use and awareness of open and interoperable cloud computing. The group brainstormed several ideas including the possibility to build on the momentum created by CloudCamp. Another topic was the ability to enable participants, from individuals and companies, both large and small, to be able to contribute to and use the results of broad community collaboration. Additionally, the possibility of a trade association or marketing association for cloud computing was discussed but no specific actions were agreed. The final topic was the need to have broader participation from the community in this discussion.

Jesse noted the conversation was extremely civil, and that each participant contributed positively to the discussion. That alone is great news to me. The atmosphere of the meeting was a key indicator to me about the likelihood that we could build open cloud standards in a cooperative, rather than competitive, fashion.

There are not a lot of details to be had about the specifics of the conversation, though it was clear that no company was willing to make any firm commitment to a specific action at this time. Just the willingness to both open future conversation to the general community and to support the organization needed to make a community targeted and productive is a great start, however.

Circumstances behind the release of the Open Cloud Manifesto on Monday morning--which was promoted by IBM and rejected very publicly by Microsoft--were not discussed. Jesse made it clear that both companies have clearly decided to put the incident behind them.

Now attention turns to Reuven's upcoming keynote at the Cloud Computing Expo, and the CCIF meeting to be held there on Thursday night. If the community embraces both the need for a trade organization and the open process proposed to establish and run it, then this may have been a very important meeting. If not, it will be another sign that the Web 2.0 era has dramatically effected industry organization and standards development.

Either way, the meeting itself signaled the acknowledgment by big business of the power of the cloud computing community. That alone is history in the making.

Update: Almost immediately after I posted this, I came across another cloud alliance that was organized to explore cloud security, thanks to Chris Hoff. Is there an opportunity here for some cooperation between the two communities (interoperability and security) moving forward?

Updated to include links to Opencloudmanifesto.org.

As widely discussed since Wednesday night's leak of its existence, the Open Cloud Manifesto--originally authored by IBM--has been released for public consumption.

This had been a difficult weekend for the document, first outed by Microsoft's Steven Martin and then leaked in its entirety by my Overcast co-host, Geva Perry, the next day.

The discussion of the document has been muted, in part because the document is not a standards declaration or contract attached to any action or entity. Instead, it serves as a simple statement of principles that almost any cloud participant would agree with--at least publicly. However, the process in which it was brought into existence has been debated ferociously and may signify a changing of the guard in the standards world.

What is perhaps more interesting, however, is the list of signatories to the document. The list below is official as of Monday morning, according to my contact at IBM:

IBM
Sun Microsystems
VMWare
AT&T
Telefonica
Cisco Systems
EMC
SAP
Advanced Micro Devices
Elastra
rPath
Juniper Networks
Red Hat
Hyperic
Akamai
Novell
Sogeti
Rackspace
RightScale
GoGrid
Aptana
CastIron
EngineYard
Eclipse
SOASTA
F5
LongJump
NC State
Enomaly
Nirvanix
OMG
Computer Science Corp.
Boomi
Reservoir
Appistry
Heroku

Note that the "big four" of cloud computing, Amazon.com, Microsoft, Google and Salesforce.com, are not signatories. However, several major players are on it, including my employer, Cisco--as well as EMC, Sun, VMware, and a host of key start-ups and established vendors throughout the industry.

There is a Cloud Computing Interoperability Forum meeting scheduled to be held Monday night in conjunction with Cloud Expo in New York City in which many, if not all of the signatories, and several that refused to sign (including Microsoft) will gather to talk about the future of cloud standards.

This could either be a historic meeting--or the final nail in the Manifestogate coffin.

The document itself is available on Scribd, or as a PDF from the official Opencloudmanifesto.org site or Perry's Thinking Out Cloud blog.

Cloud computing is the first major IT market disruption that has taken place in the world of open source software, "the wisdom of crowds" and the community collaboration revolution of Web 2.0. The concept of the cloud is trying to grow and evolve in an atmosphere in which technologists expect input on the technology they are being asked to rely on, and IT management expects input on the strategies they are being asked to adopt.

Never has that fact been more evident then in the events that have taken place over the last two days. The leaking of the Open Cloud Manifesto is a life lesson in the way that things will never be the same again.

To recap, the buzz began Wednesday night when Microsoft's Steve Martin intentionally leaked the existence of a diatribe created originally by IBM--an Open Cloud Manifesto. The industry proclamation is being supported by a laundry list of cloud service providers and members of the Cloud Computing Interoperability Forum. You can read the document on my Overcast co-host's Geva Perry's Thinking Out Cloud blog.

Since that leak, there has been a steady flow of news, retorts and excited commentary. Remember, the manifesto hasn't even been officially announced yet (look for that news to break on Monday morning)--so everything you've read so far has been pretty much who isn't participating and why.

Let me disclose right now that I was not involved in the creation of the document, nor in planning for its release, but I have been fully briefed through my employer, Cisco Systems, and the CCIF and have read the document. I planned to post my thoughts along with the others on Monday morning, and I'll still cover it in some depth at that time. For now, though, I just want to explore what I learned the last two days. (Just a quick reminder that the opinions expressed here are entirely my own, and not my employers.)

1. It's an opinion piece, not a standards proposal.

   As several people have noted, this is a big deal about something that doesn't set anything in stone, either technically or legally.

2. Those who have publicly stated that they won't sign have the most to lose.

   Microsoft and Amazon are the two cloud powerhouses that have publicly declared they will not sign the document at this time. Amazon has a huge existing install base that most other IaaS providers would like a piece of, and Microsoft is trying to hold on to an exceedingly large customer base of its own. Why should either agree to work on top-down standards to threaten that?

3. It's probably a bad idea to release even an industry opinion piece without public commentary.

   IBM, et al, left the door open for Microsoft to label the entire effort as "closed" by trying to rush to a declaration of success without allowing any public community or industry input whatsoever. Big mistake, in my opinion, because open source software has changed the game forever for technical initiatives.

   If the drivers of this initiative had simply announced that the Manifiesto draft was agreed to by the same list of companies, but was open for public commentary before being finalized, the Microsoft post would have looked silly. In fact, there is still time to declare exactly that.

4. It's what follows that is important here.

   The most important quote from the day, for me, is the following from one of the CNET reports:

   That said, Martin said Microsoft would like to be a part of the dialogue. He noted that the company was subsequently invited to a meeting of some cloud-computing participants to take place on Monday as part of a cloud-computing conference.

   "We have accepted that invitation and we will participate," Martin said. "If there is meaningful dialogue, it is something we will want to play a role in. Hopefully we will use that as a chance to restart that conversation."

   The productiveness of that meeting (and, I'm guessing, the civility) will say a lot about what will come of the manifesto. Its great that a large number of companies have (apparently) signed on to express their commitment to open cloud environments, but the actual actions initiated at that meeting--including organization, financial/people commitment, etc.--will go a long way to establishing what they can accomplished.

That being said, let me also note that I'm not convinced that a top-down formal standards approach will do anything other than repeat the mixed success of the WS-* efforts to date. Amazon's EC2 and S3 APIs are already defacto standards (see EUCALYPTUS and Sun's Cloud Compute Service), and Sun and GoGrid have also opened up their APIs in the hope they take some or all of the management standards pie. Already, businesses are out there figuring out some basic interoperability between cloud providers that matter to them: RightScale and their competitors are attacking server image portability in interesting ways, and Salesforce.com has full integration from Force.com to Amazon AWS and Facebook.

So, in the end, this declaration is a good thing in that it shows that the industry has learned that open is good. However, in the end it might not do much more than that, and we might have all gotten into a tizzy over yet another expression of what could be in cloud computing.

There were two very interesting pieces of news to come out in the last week related to the availability of relational databases in the cloud. One involved a start-up you have almost certainly never heard of, and the other involves a major player in on-premise database products.

The first was an announcement to the crowd at "Whose Cloud is It Anyway?"--a "roundtable and meet-up" sponsored by TechCrunch, held Friday on Microsoft's Mountain View, Calif., campus.

(Charles Cooper has more on the "roundtable" portion of the program. My favorite part of the afternoon was the fun comment by Salesforce.com CEO Mark Benioff; he noted the irony of hosting a cloud-computing meeting at the facilities of the vendor most disrupted by the trend.)

During the "pitch" section of the afternoon, Justin Santa Barbara of start-up FathomDB announced that the company has released to beta testing a sort of virtual managed hosting service for "standard relational databases" running on Amazon.com's Elatic Compute Cloud, or EC2, service. (There is a video of the afternoon's pitches; FathomDB starts at about 49:30.)

The start-up's current service simply allows someone to get a basic relational database management system, or RDBMS, instance (initially MySQL) up and running in minutes under its management, with services including creation, monitoring, and backup.

There has been significant discussion over the short life of the term "cloud computing" about how little it differs from concepts like managed hosting and ASPs. And there is some truth to these observations; if you really look closely, what are the key differences between EC2 and a more traditional managed hosting provider? Some would say multi-tenancy, self-service and pay-per-use (including billing and elastic capacity). With specific regard to EC2, I would tend to agree.

(I would also hasten to point out that Amazon provides some very PaaS-like services in conjunction with EC2, such as Simple Queuing Service (SQS) and SimpleDB.)

However, if this is the great "paradigm shift" of cloud computing, as offered by smart people like Krishnan Subramanian of CloudAve, then let me offer that these basic extensions to existing hosting models will be peanuts next to a shift that will create one of the most significant market opportunities since the explosive growth of the Internet itself. I'm not dealing in hyperbole here; I honestly believe that there is a clear evolutionary step to the cloud occurring well after stand-alone self-service clouds are mainstream (which they arguably are today) that will inspire massive innovation.

That game changing technology disruption will be the federation of disparate clouds, and the distribution of software, data and billing across commercial and private cloud boundaries. In other words, the introduction of secure, reliable workload mobility in an extension of the Internet itself--an "Intercloud", so to speak.