A maturity model for cloud computing
One of the really difficult aspects of cloud computing for most established IT organizations is the fact that the move to clouds, even private clouds, is not a simple, intuitive one. Replacing the bulk of both technology and process with a focus on capacity as a service--an automated, self-administered service--results in many organizations "experimenting" with the cloud, but few pushing any barriers. To make matters worse, we are in that wonderful "discovery" phase of a technology, where there are few if any guides to how to do it right, with minimal risk, and those that do exist are generally personal opinions, not "burned in" recipes for success.
This post does not pretend to be such a recipe. However, over the course of the last several months, culminating in some great conversations with some really smart people the last few weeks, I've come to realize that there is a basic maturity model for moving from data center consolidation architectures to true open market cloud architectures.
Remember maturity models? They've been around for some time, but a couple of years ago there was a small burst of creativity among system integrators and analysts alike, and maturity models were defined for a variety of IT subjects, ranging from business processes to technology architectures, such as SOA. The basic idea was to lay out some milestones, or even "gateways", to be achieved by IT as they worked towards achieving some idealized computing or process goal.
To that end, below is a simple five phase maturity model that I and others believe describes the stages of evolution for an enterprise data center trying to achieve cloud Nirvana:
(Credit:
James Urquhart)
At a very high level, each step of the model breaks down like this:
- Consolidation: is achieved as data centers discover ways to reduce redundancy and wasted space and equipment by measured planning of both architecture (including facilities allocation and design) and process.
- Abstraction occurs when data centers decouple the workloads and payloads of their data center infrastructure from the physical infrastructure itself, and manage to the abstraction instead of the infrastructure.
- Automation comes into play when data centers systematically remove manual labor requirements for run time operation of the data center.
- Utility is the stage at which data centers introduce the concepts of self-service and metering.
- Market is achieved when utilities can be brought together over over the Internet to create an open competitive marketplace for IT capabilities (an "Inter-cloud", so to speak).
Virtualization is actually a tool for consolidation, and it is probably its most applied function at this point, but consolidation can be achieved other ways, including storage consolidation and use of denser compute hardware (e.g. blades and dense switches). Consolidation certainly provides return on investment in most cases, but it doesn't really add anything to the ease in which hardware is provisioned and assigned.
Server virtualization is certainly a starting point for abstraction, but unless used specifically as the target representation of workload in the data center, and coupled with things like live motion and storage virtualization, it doesn't go far enough. Network abstractions have been around for a while (e.g. VLANs), but they have frequently been underused as an abstraction, instead focused on a static method of consolidation. I think commercial clouds today are almost all providing abstraction as a service, and some move somewhat beyond that. Unfortunately, while abstraction enables dynamism, it doesn't maximize impact on system administrator productivity.
Provisioning automation has been around for a while, but real maturity in automation requires things like pooling for rapid reallocation of compute resources, run time response to capacity demands, trouble ticket response automation (or elimination of trouble tickets for most automated response scenarios), and integrated system management and measurement. While automation can greatly reduce your operational expenses in the data center, there is much more that an IT organization can do to align operations to business needs.
Creating a utility requires IT to get out of the way of the business units looking to allocate IT resources for various initiatives and core business requirements; in other words, to allow businesses to serve themselves. However, in order for self-service to work for the overall enterprise, the business units need feedback as to the cost of those resources; this is why metrics are so important to the utility model. Once both self-service and metering appear and are used consistently throughout IT, the data center starts looking like a true utility--though a very monopolistic one.
Achieving an open marketplace is essentially cloud computing nirvana, and the ideal to which most enterprises should logically strive to achieve. Cloud providers, on the other hand, are likely to want to hold this off for a while, as it will create commodity markets for certain types of computing resources. In the end, I believe this open market is inevitable, as the economics are just too powerful.
From the evidence I can find, it appears that most competent IT organizations have managed some form of consolidation in the last five years, and many are well on their way to having a handle on how to use abstraction to add value to IT. Many forward thinking organizations are already experimenting or even implementing fairly comprehensive automation for scalable workloads, with some even going farther with plans to automate all infrastructure. Almost no one but cloud providers (and a few dev labs) are at Utility (and I should point out that just having a portal where users can request capacity to be manually allocated later doesn't count.) A true cloud marketplace is but a gleam in any technologists eye at this point.
Let me know what you think about all of this. Where do you feel your organization is with respect to cloud maturity? Are you eager to climb the maturity ladder, or does it just not matter to you or your organization?
James Urquhart is a seasoned field technologist with almost 20 years of experience in distributed systems development and deployment, focusing on service-oriented architectures, cloud computing, and virtualization. James is currently market manager for the Data Center 3.0 strategy at Cisco Systems, though the opinions expressed here are strictly his own. He is a member of the CNET Blog Network and is not an employee of CNET. 
Brings the cloud a step closer to cloud adoption!
Best.
Alain
I especially like "Achieving an open marketplace is essentially cloud computing nirvana, and the ideal to which most enterprises should logically strive to achieve" but of course, I'll add onto this that such a nirvana will only be practicably achievable (without loss of strategic control and pricing competition to a technology vendor) if the standards which it is built upon are operational open sourced pieces of code rather than specifications (it's an old post from July'07 but is covers the basics - http://blog.gardeviance.org/2007/07/competition-not-greed-is-good.html)
Such open sourced standards (the equivalent of open design patterns, such as the open SDK for GAE) will allow competition and service innovation through implementation. The standards themselves will insure portability, the open marketplace you describe and hence consumer innovation through componentisation.
As for "I believe this open market is inevitable, as the economics are just too powerful" - well it's either going to be open or we will see government intervention in the field to force it to behave in such a way.
You can create the illusion of open marketplaces with proprietary stack but it is no more than an illusion (see http://blog.gardeviance.org/2008/03/timeo-danaos-et-dona-ferentes.html)
Anyway, good post.
As far as maturity models go - well, you've seen this before and though it's tongue in check, it does have some serious point to it, as always. (see http://blog.gardeviance.org/2008/02/usaasmm-what.html)
Congratulations on the new gig by the way. I'm looking forward to reading more and seeing how this develops.
I'm somewhat surprised, though, to find that mobility isn't included as a separate stage in your model, because without it the later stages in your model aren't possible. Perhaps you intended abstraction to include mobility, but the description only mentions virtualization and vlans.
Just my two cents.
Anyone else have a "feature" of the cloud they'd like to see explicitly called out?
James
Great to hear from you.
Actually, I don't say that open source is the only answer as you can provide the same effect with a proprietary stack. However, what I do say is that open source is the only answer which doesn't involve handing strategic control and pricing competition over to a technology vendor. It's for this reason that I argue that open is the right answer.
I know I sound like a stuck record. I've been going on about this for several years (http://blog.gardeviance.org/2006/10/open-sourcing-zimki.html) and it's almost 17 months since we first crossed paths (http://blog.gardeviance.org/2007/07/competition-not-greed-is-good.html).
How time zooms by.
I still firmly hold the view that portability between providers (at all levels of the computing stack) will become essential for the "cloud" to grow and that in practice such portability will depend upon standards which are operational pieces of open sourced code.
Bert, really good point about the mobility issue as this is an essential part of a marketplace but also an essential part of abstraction and the decoupling of instance to hardware. I did actually think this was part of James' abstraction level.
One final point, obviously whilst the model is a good general concept, the actual development will probably not be so linear.
The development will certainly not be so linear for the market as a whole. However, I contend the most successful companies will go through exactly these stages, at least at the project level. Utility without abstraction and/or automation is possible, but broken at its core (you end up "reserving" physical machines for a specific purpose).
It's possible to speed up the process by acquiring a platform that includes abstraction, automation and utility; doing all of them at once. However, as those of us who have sold those platforms can attest, that's a "big deal" to do at anything larger than the 100-200 server project level. Things go much better at a large scale with clients that already manage abstractions; even better when they have experimented with automating management of those abstractions.
James
- by bethpinson March 2, 2009 1:25 PM PST
- I enjoyed reading your points and also taking in your maturity model. Very interesting. I am interested to watch Cloud Computing mature within large business models. Walter Pinson has a great post on the immaturity of Cloud Computing and how Amazon, Google and Microsoft are trying to figure out how to gain further returns on their investments. This is all very thought provoking stuff to me. <a href="http://www.walterpinson.com/index.php/2009/02/is-cloud-computing-like-teenage-sex/">Is cloud computing like teenage sex?</a>
- Like this Reply to this comment
-
(9 Comments)