Cloud computing and the big rethink: Part 2

In the opening post of this series, I joined Chris Hoff and others in arguing that cloud computing will change the way we package server software, with an emphasis in lean "just enough" systems software. This means that the big, all-purpose operating system of the past will either change dramatically or disappear altogether, as the need for a "handle all comers" systems infrastructure is redistributed both up and down the execution stack.

The reduced need for specialized software packaged with bloated operating systems in turn means the virtual server is a temporary measure; a stopgap until software "containers" adjust to the needs of the cloud-computing model. In this post, I want to highlight a second reason why server virtualization (and storage and network virtualization) will give way to a new form of resource virtualization.

I'll start by pointing out one of the unexpected (for me at least) effects of cloud computing on data center design. Truth be told, this is actually an effect of mass virtualization, but as cloud computing is an operations model typically applied to virtualization, the observation sticks for the cloud.

Today's data centers have been built piecemeal, very often one application at a time. Without virtualization, each application team would typically identify what servers, storage and networking were needed to support the application architecture, and the operations team would acquire and install that infrastructure.

Specific choices of systems used (e.g. the brand of server, or the available disk sizes) might be dictated by internal IT "standards," but in general the systems that ended up in the data center were far from uniform. When I was at utility computing infrastructure vendor Cassatt, I can't remember a single customer that didn't need their automation to handle a heterogeneous environment.

But virtualization changes that significantly, for two reasons:

• The hypervisor and virtual machine present a uniform application programming interface and hardware abstraction layer for every application, yet can adjust to the specific CPU, memory, storage, and network needs of each application.

• Typical virtualized data centers are inherently multitenant, meaning that multiple stakeholders share the same physical systems, divided from one another by VMs, hypervisors, and their related management software.

So, the success of applications running in a virtualized environment is not dependent of the specialization of the underlying hardware. That is a critical change to the way IT operates.

In fact, in the virtualized world, the drive is the opposite; to create an infrastructure that drives toward homogeneity. Ideally, rack the boxes, wire them up once, and sit back as automation and virtualization tools give the illusion that each application is getting exactly the hardware and networking that it needs.

Now, if the physical architecture no longer needs to be customized for each application, the question quickly becomes what is the role of the virtual server in delivering the application's needs. Today, because applications are written against operating systems as their deployment frameworks, so to speak, and the operating systems are tuned to distribute hardware resources to applications, virtual machines are required.

But imagine if applications could instead be built against more specialized containers that handled both "glue" functions and resource management for that specialization--e.g., a Web app "bundle" that could deal with both network I/O and storage I/O (among other things) directly on behalf of the applications it hosts. (Google App Engine, anyone?)

A homogeneous physical architecture simplifies the task of delivering these distributed computing environments greatly, as there is a consistency of behavior from both a management and execution perspective. However, as it turns out, a homogeneous virtual container environment has the same effect.

So, if the VM isn't hiding diversity at the hardware layer, or diversity at the software layer (which is hidden by the "middleware") what is its purpose? Well, there is still a need for a virtual container of some sort, to allow for a consistent interface between multiple types of cloud middleware and the hardware. But it doesn't need to look like a full-fledged server at all.

Thus, the VM is a stopgap. Virtual containers will evolve to look less and less like hardware abstractions, and more and more like service delivery abstractions.

In my next post, I want to look at things from the software layers down, and get into more detail about why applications will be created differently for the cloud than they were for "servers." Stay tuned.

[meh130]

I disagree with the comment "The hypervisor and virtual machine present a uniform application programming interface ... for every application." APIs (or at the OS level, more correctly ABIs) are fundamentally the job of the OS, not the hypervisor. This is the role of the "Just Enough OS", or "JEOS" model of slimmed down OS on a hypervisor. Of course, with a rich hypervisor, you could program directly on the hypervisor, but now you end up with a multiuser OS, not a hypervisor. I would agree the hypervisor and virtual machine present a uniform platform and hardware abstraction layer for every application.

You bring up excellent points. There are three layers in the application stack affected by the virtualized model. The hardware layer (CPU ISA, or hardware platform), provided by the hypervisor and VM, the OS layer (ABI, i.e., POSIX, etc.), and the middleware layer (API, i.e., Java or .Net). Clearly, the OS layer loses much of its role (hardware management, drivers, etc.) to the hypervisor. But how "thick" or how "rich" each layer should be will be debated for some time. An appliancized middleware stack, including a JEOS, and running on a VM, almost eliminates the OS. But, a clustered middleware stack may take on platform roles (clustering, etc.) currently associated with a rich hypervisor platform.

The hypervisor could get relegated to a HAL, or the OS to a shim, or something else. Likewise, traditional OS roles which have migrated to the hypervisor, such as storage volume management and file systems, could easily move out of the server and into the network as either a network based storage management solution like IBM's SVC, or network based file systems such as NFS.

[jprescott]

Seems like we've been here before. IBM VM on 3090, etc. mainframes. The hypervisor becomes an operating system. You still have a software layer that provides a consistent abstraction of the hardware to an application to make it easier for the application writer. That is what an operating system fundamentally does. Now, the point is that the nature of the "operating system" is going to change to become much more "class of application" specific (an "web" operating system, a "compute" operating system, etc.) rather than the more general-purpose operating systems we use today, that is different than saying the "operating system" is going away. Even if you go to an "appliance" architecture where the hardware and support software are tailored to a specific application class, there will most likely be an ABI layer that would be identified as an "operating system" to make it easier to port and maintain application software.

[lenghui]

Good article, but I somewhat disagree with the author. VM could be an effective part of a cloud solution, rather than just a gap-stopper. VMs are especially crucial for leveraging under-utilized hardware and for disaster recovery. They can be used to enhance a cloud but are certainly not required.

One important aspect that "hardware architects" often forget is, just because you have a cloud environment does not mean all applications should be cloud-based. This is a common golden hammer problem in most enterprises. I am all for cloud computing, but it comes with baggage too - longer development time, more layers translate to longer deployment process and increased complexity, potential limitation by a cloud service/API provided, possible performance issue, and potential bottleneck problem (Your app can only be as good as the weakest link in the cloud).

[mmullany]

I don't think you're factoring time into the equation properly here. The idea that you set up your datacenter ONCE with homogenous hardware is just not how it happens.

People replace parts of their hardware infrastructure on a yearly basis, they switch vendors, they upgrade firmware, they do an acquisition and you have to integrate a completely different hardware/storage base into your operations. Without hardware virtualization, this is a mess.

Also the great weakness of the Google App Engine approach is that people always need to add something that doesn't fit into a software level container for real-life applications.

[LeitM]

James,

Can you elaborate what you mean by the following?

1. But virtualization changes that significantly, for two reasons:
The hypervisor and virtual machine present a uniform application programming interface and hardware abstraction layer for every application, yet can adjust to the specific CPU, memory, storage, and network needs of each application

Question: How is this adjustment done specific to each application today?

2. But imagine if applications could instead be built against more specialized containers that handled both "glue" functions and resource management for that specialization--e.g., a Web app "bundle" that could deal with both network I/O and storage I/O (among other things) directly on behalf of the applications it hosts.

Question: If these specialized containers can pretty much do whatever is best for their application, how is the overall physical resource management orchestrated? Who decides across all these application containers how the physical resources should be allocated?

3. Question: The current optimization being provided by Intel/AMD on their processors for virtualization - how will it change in this new scenario?

LeitM

[SenorCloud]

Great article. I have to say, i agree over-all with your thesis. Cloud computing will change the way we package server software. Here's a fun video for all you cloud computing enthusiasts out there, it's really funny: http://www.youtube.com/my_videos

Stay safe, my friends. - Señor Cloud

[abawcom]

There is a post by Joel on Software that seems relevant to this discussion: http://www.joelonsoftware.com/articles/APIWar.html

In this he article he writes: "I first heard about this from one of the developers of the hit game SimCity, who told me that there was a critical bug in his application: it used memory right after freeing it, a major no-no that happened to work OK on DOS but would not work under Windows where memory that is freed is likely to be snatched up by another running application right away. The testers on the Windows team were going through various popular applications, testing them to make sure they worked OK, but SimCity kept crashing. They reported this to the Windows developers, who disassembled SimCity, stepped through it in a debugger, found the bug, and added special code that checked if SimCity was running, and if it did, ran the memory allocator in a special mode in which you could still use memory after freeing it."

The wisdom of the reference above is that the hammer of compatibility can either build new technology or destroy it. Looking into the crystal ball of how things will operate in the future is absolutely necessary; otherwise no one would come up with better ways to do things. I agree that a new definition for "application" is coming and that virtualization/cloud computing are the youthful hands shaping this definition but if we want the millions of existing applications to be operable within the cloud we must be mindful of how they work today without requiring their restructuring based on technology that doesn't exist today. As John (Cisco) and others have stated security is one of the biggest roadblocks to the adoption of cloud computing and solving those problems is a huge hurdle for the broad adoption of cloud computing ->today<-. Some believe the only way to solve these security problems is by changing how applications are defined and run. This reader thinks there is a faster way to get there: http://blog.reflexsystems.com/2009/08/17/iaas-cloud-drift/