While most readers probably think "portability or mobility" when they think of cloud interoperability, the vendor community sees a shorter-term opportunity in standardizing the operation of clouds and cloud infrastructures. It's not that vendors don't care about image portability; it is an especially critical opportunity for so-called "cloud operating system" vendors.
However, the cloud operations opportunity--building a full-featured operations API and user interface for a cloud--is a daunting task, requiring tools for provisioning, management and monitoring, among others.
(Note that I am calling the term "operations" tools, not "management" tools. It may seem like semantics, but when I published part 1 of this series, one of the systems management tools vendors quickly came back and said "hey, that's us!" Unfortunately, I'm not talking about monitoring and responding to problems with specific cloud payloads here, I'm talking about the interfaces you use to provision and control the clouds themselves. Of course, there is often overlap between the two concepts, just to make things...er...cloudier.)
In some ways, cloud operations interoperability is your basic API standardization story; allow several different independent management tools interact with several different independent cloud platforms and providers. Create a loosely coupled market dependency between the tools you use and the systems you manipulate with those tools. This is always a desirable goal in any open software marketplace.
However, there is a little more at stake here than simply enabling tools to be used across platforms. Consider this scenario: an enterprise has a distributed application system that runs components on two different major cloud provider infrastructures, relies on services running on two or three more, and stores data in their local data center. What would be the most desirable way to address the management of this system?
I would argue that you want as few "panes of glass" (i.e. disparate management applications) as possible, preferably one. To do that, you need standards that the single, all-encompassing management platform can rely on to access and manipulate all of the cloud platforms involved in the system.
In fact, you probably want to combine SOA and cloud management into a single pane of glass, so you probably want the standards to cover just about everything in the computing stack, from servers, storage and networking to specific application components and data stores.
Today, the focus is on providing a unified API for Infrastructure as a Service operations. In addition to standardizing how systems are provisioned, when they are active and what policies apply for situations like component failure or load spikes, it is also critical that this API unifies the way in which images are imported and exported from each provider's platform. A cloud operations API needs to cover as much of the system life cycle as possible, including provisioning and deployment.
There is so much effort being made in this space right now, by so many groups, that it is at times a little overwhelming. Luckily, I found a resource that has helped me organize not only the predominant operations API effort, but also the image/data and application/service interoperability classes. Believe it or not, it is a wiki dedicated to cloud efforts in the federal government market. (I'm telling you, the feds seem to be way ahead of the pack when it comes to organizing cloud activities these days.)
The wiki lists the following key efforts for "Governance and Management":
Management of Virtualized Resources - DMTF (http://www.dmtf.org/initiatives/vman_initiative/)
Open Cloud Standards Incubator - DMTF (http://www.dmtf.org/about/cloud-incubator)
SLAs -at SOI (http://sla-at-soi.eu/)
Compliance - Industry organizations (e.g. HIPAA, PCI) (http://www.hipaa.org/)
I was surprised to see three organizations missing from this list: the Cloud Computing Interoperability Forum (CCIF), the Open Cloud Consortium (OCC), and the Open Grid Forum's Open Cloud Computuing Initiative (OGF OCCI). It turns out the CCIF and OGF are acting more as organizing entities here, and the OCC is working standards around data and application interoperability, which I will talk about in part 4 of this series.
There is a "Cloud Standards Summit" planned for July 15th in Arlington, Va., that may clear further organize "who's doing what" in this space. I think the fact that such a meeting is taking place at all at this point is phenomenal. It is clearly an artifact of cloud computing's distinction as the first major IT disruption to take place in a world that assumes openness and transparency, thanks in large part to open source software.
As much as anything, cloud computing has borrowed from open source in terms of its governing principles, which could well be open source's lasting contribution to the cloud. It took a few decades of Microsoft dominance to really get the open-source movement in full swing, but it only took a few months for things like the Open Cloud Consortium to spring to life. Open source has taught us to expect openness by default. The cloud is no different.
Earlier, I talked through the three key categories of cloud interoperability as I see them, and the promise that each holds for increasing the agility and value of IT. One of those categories was Image/Data interoperability, which I defined as follows:
This is the one that most people assume when they say "cloud interoperability." How do you define a virtual server image, or a Java application, or a customer relationship management (CRM) database, such that it can be deployed on another host, often a competitive host, without modification?
In reality, Image/Data interoperability can be further divided into two subcategories:
Portability: The ability to move an image in a "down" state, and boot it at its destination. The image, in this case, can be as simple as a file system or as advanced as an Open Virtualization Format (OVF) portable VM image with sophisticated metadata. This can be considered a "stateless" move.
Mobility: The ability to move a live compute workload without losing client connections or in-flight state. VMotion is an early example of workload mobility, but the future promises mobility of "virtual containers," which incorporate one or more VM, network policy, storage policy (which may or may not contain actual data), and additional metadata around dependencies, automation policies, etc.
The problem facing the market right now is that most people assume that mobility is the target everyone is shooting for today. It's true that in the long term, everyone I've spoken to is targeting mobility as a way to create opportunities for cost savings and new business models. I've written about this before.
However, it is also a sobering fact that mobility across data center boundaries (especially across so-called "Layer 3" boundaries) will be some time in coming. The issue isn't simply networking challenges, though they are big--for example, global IP address portability, latency, security, etc. Other challenges are equally as daunting, such as how to manage data in a dynamic deployment environment, how to maintain access from mobile VMs to static resources, etc.
The truth of the matter is that global live workload mobility is still an early research and development project. Unless, of course, some start-up comes along and proves me wrong...
That being said, there is an algorithm for simulating mobility that is very compelling today, and I believe will be used in a wide variety of applications in the coming years. In fact, it is already common in Amazon EC2 deployments, and you might already be using it within your own data centers today.
The idea is to take advantage of the portability of images to replicate compute workloads across all of the data centers in which you may want it to run. Then, you set up your global load balancing to distribute load to one or a few of those instances. When "mobility" is needed, you just change the load-balancer configuration to redirect to instances in other data centers.
I call this scenario "faux mobility," and it is for all intents and purposes a stateless mobility algorithm. Faux mobility is what you might use to distribute risk across multiple availability zones in Amazon Web Services. It may also be the way you distribute risk across multiple cloud providers. One of my favorite uses is an algorithm I've heard called "whack-a-mole," in which an application is kept running in a single data center at a time, but if availability to data center A is interrupted, load balancers send subsequent requests to data center B--which now becomes the primary data center for the application. This can be repeated over and over again as events require.
While faux mobility is extremely useful, it won't be the game changer that live mobility will be, as the former requires images to be stored in all applicable data centers/cloud providers before an event occurs. Live mobility will change things drastically precisely because the image (or "virtual container" of images) can be moved to a provider whether the provider knew of the image beforehand.
Finally, it is interesting to note that, while most cloud customers probably think of portability and/or mobility as "cloud interoperability," it is not the primary front in the standards effort under way today. That would be management interoperability, where there are several efforts already under way. I noted several of these in part 1 of this series, but I will cover the issues in more depth in part 3.
The announcement by the Distributed Management Task Force (a systems management standards group) that they were going to build an "incubator" to research and develop interoperability standards for cloud-computing management is just the latest step in an accelerating effort to unify "the cloud." Everyone is getting involved, from virtualization vendors to public cloud providers to the major enterprise IT systems vendors.
But what exactly is cloud interoperability, and what exactly are each of these efforts addressing? Where are the standards going to be created, or (perhaps more importantly) where is the technology going to come from?
I thought it would be useful to give my current understanding of the space, and to give you my 100,000-foot view of the cloud interoperability landscape today.
Cloud interoperability is one of those terms that actually covers more ground than it would appear to at first. However, at its heart, it refers to the ability of customers to use the same artifacts--management tools, server images, etc.--with a variety of cloud-computing providers and platforms. Something like "I want to reuse my Linux image from Amazon on Slicehost without changes" requires standards and agreements to enable Slicehost's platform to read Amazon AMI files.
I usually divide the concepts into three interoperability targets:
Application/Service
- Definition: Some might argue with me that this is technically not cloud interoperability, but I think the cloud forces application developers to reconsider how they couple applications together. Thus, traditional application connectivity and integration issues come into play for me. The key question I ask myself in this category is how I can loosely couple applications and services while maintaining some resiliency to changes in connectivity and location. If I were to live-motion a portion of a distributed application system, would the cloud provide services to help me maintain the connections and contracts required for the application to remain viable?
- Examples: There are a tremendous number of "traditional" distributed application standards and technologies at play here. Service-oriented architecture diehards SOAP and REST, for example, combined with good ol' DNS, will enable a lot of systems to "rediscover" dependencies as needed. More sophisticated integration, through innovative services like cloud integration from Boomi or Cast Iron, which allow the Internet to be seen as an enterprise service bus (ESB) of sorts.
Management
- Definition: This is where a lot of early work is happening in cloud interoperability. What are the APIs by which a management application can control multiple cloud environments--both public and private? This includes how images can be delivered between providers (but not how they are packaged--see below), how servers and/or applications can be started and stopped, how storage can be manipulated, etc.
- Examples: At this point, the Amazon Web Services APIs are looking like de facto standards in the management tools space right now. However, GoGrid has released their API under creative commons licensing, and there are at least three competing "open" efforts in this space, including the Cloud Computing Interoperability Forum, the Open Grid Forum's new Open Cloud Computing Interface Working Group, and the aforementioned DMTF incubator.
Image/Data
- Definition: This is the one that most people assume when they say "cloud interoperability." How do you define a virtual server image, or a Java application, or a Customer Releationship Management (CRM) database, such that it can be deployed on another host, often a competitive host, without modification? For virtual machines, the DMTF's OVF standard is fairly far along in defining not only how to represent a "raw" server image, but also a "live" machine; the latter being necessary to move a workload across any network connection, even the Internet itself, without losing a client connection.
- Examples: The DMTF's Open Virtualization Format (OVF) standard defines a format for describing a virtual machine in terms that can be interpreted by a variety of virtualization platforms, such as VMWare's vSphere and RedHat's KVM distribution. Interestingly, it looks like Google App Engine is making a concerted effort to keep Java applications developed there portable to a variety of middleware options.
Of course, the politics of interoperability will be playing out for some time yet. All of this activity is by no means a guarantee that we will see any interoperability in the near future. However, I hold out hope that at least the application/service and management interfaces can be defined and adopted in a few short years. I also believe that image/data portability can be achieved in the near term in many cases.
Image/data mobility, however, is another story. I'll leave that for part 2.
The Cloud Interoperability meeting prior to Cloud Connect in Mountain View, Calif., last week was a very interesting petri dish of some of the best and brightest in the cloud-computing marketplace.
There certainly was a quorum of companies represented (though Amazon.com couldn't make it at the last minute, and Microsoft never replied to the invitation). There also, as you might imagine, was no shortage of opinion on how to proceed.
As you might imagine in such a situation, most of the day was taken by attendees expressing their personal visions of cloud interoperability and standards building, only to boil next steps down to developing a taxonomy and sorting out a small list of the most pressing concepts to be explored. A wiki was proposed, and I will share the URL when I get it.
Here is the whiteboard at the end of the day (artistry courtesy of David Berlind, one of the founders of the event):
... Read more- prev
- 1
- next
