The Pervasive Data Center

More and more of our computing happens through applications and Web sites out in the network. It's in the "cloud" to use the current trendy lingo.

One consequence is that we're starting to look at our clients differently. That's because they're increasingly a sort of window into the cloud rather than devices that run a lot of application-specific code and store a lot of application-specific data locally. Clients can therefore be "thinner," which is to say loaded with less software and less tailored to the needs and wants of a given user. Resources and customization live out in the network instead.

Even with more conventional operating systems such as Windows, Linux, or OS X, running applications in the network reduces the time spent installing and upgrading applications on our proliferating collection of clients. Google's Chrome OS takes the concept to the next level and essentially reimagines the client OS for a cloud world.

However, the real world is messier and more complicated than "Just run everything in a browser." That's true today and will almost certainly be true to at least some degree next month and next year. Ultimately, this question of how thin clients can become as a practical matter is going to play a big role in how accepted certain models of computing will become.

To illustrate, consider a PC that is today mostly used to go online. There's more than just an OS and a basic browser involved.

There are plug-ins and extensions for the browser. There's probably an IM client; Meebo is a Web-based alternative but most people run a local client. If you use Twitter, there's a good chance you run an application like TweetDeck or Seesmic, which may in turn require Adobe's AIR runtime. Third-party media applications such as Apples iTunes are commonplace. Google Earth, Windows Live Writer...This list goes on--and will vary by user--of the applications and components that have to be installed and updated for even a rather bare-bones PC configuration.

And that's before we even broach device drivers or other software that may be required to connect a camera, a microphone, or some other peripheral.

My overarching point here is not that a thinner client model is uninteresting. I strongly believe that it is meant not to replace traditional fat clients but to augment them. Today, I have a notebook that is essentially used only to go online yet I still have all the administration associated with a full-blown PC.

However, the challenge for Google and others is to steer a course that creates an "Internet computer" that is legitimately better in that role than a full-fledged PC while retaining sufficient customization. Application stores may be part of the answer. HTML 5 will likely also help by making browsers more capable of running applications.

Whatever the specific technical solutions though, the answer will involve a lot of careful thought about balancing simplification and flexibility.

This is the time of year to take stock in where high-performance computing (HPC) sits and where it is headed. That's because the SC09 conference is taking place in Portland, Ore., this week and it's the biggest HPC conference around.

SC is an odd duck as conferences go. Last year it had more than 10,000 attendees and, yet, it's a largely volunteer-organized event in a world where trade shows of this scope are packaged by conference specialists or some specific corporation. Think the much-renamed LinuxWorld  (run by IDG) or VMworld (run by VMware).

"SC" comes from supercomputing. Today's large computer complexes are typically not supercomputers in the sense of a specialized architecture only suitable for a specific type of technical computing. Rather, as Ashlee Vance notes in The New York Times, "The supercomputing world was long dominated by systems that required specialized chips, memory systems and networking technology. But about 10 years ago, researchers realized they could link thousands of cheaper machines running on mainstream chips and achieve pretty solid performance."

Thus an HPC event is no longer about supercomputers per se (although the term is still used as a convenient moniker for a collection of resources managed as a single entity in a single location). Rather it's about the computing components, the interconnects, the storage, and the software that ties everything together and the applications that run on top.

The Top500 nicely illustrates the evolution of HPC over time. This list, released twice annually, ranks the largest publicly acknowledged supercomputers--as the term is used today--on the basis of a somewhat simplistic, but objective, benchmark. The Top500 entries are certainly not typical of mainstream HPC; they're the biggest of the big. But they nonetheless provide some quantitative insight into important trends.

The newest iteration of the list was released Friday. There were no striking departures from the trends of the last few years, but there was some continued evolution that's worth taking note of.

The continued rise of InfiniBand. InfiniBand is a system interconnect that offers a higher performance alterative to the ubiquitous Ethernet. Although its initial backers envisioned a broader role for the technology, it's settled nicely into HPC and, to a lesser degree, back-end commercial data center functions like database clusters where low latency and high bandwidth are also paramount. (The Sun/Oracle Exadata appliance uses InfiniBand for example.)

(Credit: TOP500.org)

InfiniBand's initial growth in HPC wasn't so much about displacing Ethernet as it was displacing the fractured collection of high-performance interconnects that preceded it. Myricom's Myrinet and Quadrics' QsNet were the most common of these, but there were many. This year InfiniBand is deployed on 181 of the Top500, a 28 percent increase from a year ago.

That's a striking increase clearly. But what is perhaps more striking is that about half that increase came at the expense of Ethernet rather than mopping up a variety of older or proprietary connection technologies. This shift started between 2007 and 2008 but was even more pronounced this year.

It's certainly possible that the next 10GbE generation of Ethernet, which today is essentially absent from the list, could again push Ethernet's numbers higher. However, whatever the specific technology, the message that I take away is that large computer clusters are starting to favor more optimized interconnects even if they and the components they connect are largely off-the-shelf.

And we see an analogous trend with the proliferation of blade servers as well. Blades, a more modular and pluggable approach to system design, have proven popular in many enterprises and midmarket companies, in part, because they help bring together computing, storage, and networking technologies into a single integrated whole. That type of integration isn't of much interest in HPC. Rather, blades play to HPC by offering high densities and reducing cable count and complexity.

In fact, among x86 servers at any rate, dominance is not too strong a word to describe the presence blades in the Top500. Consider just one vendor, Hewlett-Packard. HP has 208 ProLiant systems on the list. A full 203 of these, almost 98 percent, are ProLiant c-Class blades.

Collectively, these trends suggest what might be thought of as a trend toward building optimization around standardization. In the main, especially as one moves down from the very top of the list, the Top500 is composed mostly of systems using mainstream technologies such as x86, Linux, and standard interconnects. Clusters are the dominant architecture.

But we're increasingly not seeing mere rackmount servers connected by Gigabit Ethernet. As the systems on the Top500 list grow in capability, we're seeing more focus on how they're packaged, powered, and connected.

On the one hand, vendor analyst events are a good opportunity to spend focused time diving deep into individual products, roadmaps, and corporate initiatives. On the other, they're a useful forum for getting the feel of a company's overall zeitgeist in a way that narrower discussions don't. EMC's event, held last week in Franklin, Mass., was no exception.

(Credit: EMC)

Perhaps the single thing that struck me most about the event as a whole was the full integration of VMware into the discussion as a whole. I've been following both companies since before EMC acquired VMware in 2003. In the years since, although there were the obligatory nods to joint development work and "better together," VMware aggressively maintained a distance that was hardly limited to the 3,000 miles between VMware's Palo Alto, Calif., headquarters and EMC in Massachusetts. VMware's presence at EMC analyst events was largely relegated to a few off-hand mentions and perhaps a desultory breakout session given by a junior marketing person.

This year couldn't have been more different. VMware was very much woven into just about every discussion and one of VMware's senior technologists shared a panel with representatives from EMC and Cisco Systems. One thing that has changed, of course, is the ouster of VMware founder and CEO Diane Greene in 2008. It was Greene who most vocally kept EMC at arm's length. It's also the case that virtualization is increasingly at the center of everything that EMC does, so how could VMware not be an integral part?

This pervasive virtualization theme carried through to EMC VP Jon Peirce's discussion about EMC's internal IT infrastructure as well. EMC IT is using VMware to virtualize as much as possible. This includes doing database testing on a Cisco Unified Computing System (UCS) in advance of a planned migration off Sun E25000 UltraSPARC-based servers.

An initial Virtual Desktop Infrastructure (VDI) deployment also uses UCS in the form of a vBlock--a preconfigured package that combines products from Cisco, EMC, and VMware. EMC has about 200 users on VDI today and expects to roll out to several thousand next year starting in their Franklin facility. VDI and associated forms of desktop virtualization are a favorite technology of CEO Joe Tucci, who would like to move toward a platform-agnostic client strategy.

The ultimate goal is what sometimes goes by BYOPC (Bring Your Own PC), in which employees provide their own notebook computers, perhaps purchased with the help of a stipend. Even today, many of the EMC execs at the event were sporting Macs, even though IT doesn't officially support them.

Another hot topic at the event was multi-tier storage, in this case automatic storage tiering that intelligently moves data between Flash-based storage and conventional disk drives. EMC's technology here is called FAST and will roll out on Symmetrix V-Max arrays.

Flash drives can be much faster than SATA disks--or even high-performance Fibre Channel drives--but they're also much more expensive on a per-GB basis. The idea behind FAST is to automate the placement of data based on the way its accessed. For example, a database index that is frequently read and written to will migrate to high performance flash while older data that hasn't been touched for a while will move to slower, cheaper disks.

Disks being used to store rarely accessed archival data can even be deduped, compressed, and even spun down to reduce overall data center power consumption. Tape isn't part of this vision; Tucci opined that "Backup to and recovery from tape is dead."

The idea of storage tiering isn't new. Hierarchical storage management (HSM) has been around for well over a decade. However, in practice, it's mostly ended up being about moving old files to tape for archive purposes. (EMC itself has a product in this vein: Legato DiskXtended.) FAST is something more transparent and more dynamic.

There are analogs between FAST and the storage pooling that is part of Sun Microsystems' ZFS filesystem. EMC argues that the function belongs on the storage device rather than the server because the array is where data access from multiple systems and applications come together.

It's unsurprising that EMC wants storage to be at the center of things. This is a company, after all, whose tagline is "where information lives." It is, however, worth remembering that this is a different lens through which to view the world than system vendors tend to choose--and, for that matter, than VMware chose historically.

Although VMware got its start with a desktop virtualization product aimed at developers, the company today is best known for bringing server virtualization to the mainstream.

Creating multiple virtual servers on a single physical system lets IT departments consolidate applications onto fewer computers and thereby cut costs. Over time, server virtualization has also enabled a variety of products and approaches that can simplify IT operations and generally make data centers more flexible.

VMware has continued to invest in virtualization aimed at the client. This includes client-side hypervisors such as its original VMware Workstation product. However, products and technologies associated with delivering applications and user desktops to the client are really the main focus.

Application and desktop delivery sometimes makes use of client hypervisors but it's a largely separate category of technology that's fundamentally about centrally managing user applications and/or operating-system images. In VMware's case, virtualized desktops fall under the VMware View name.

On Monday, VMware announced VMware View 4, the latest version of its virtual desktop portfolio.

Much of VMware's development focus with View 4 was in the area of the user experience--that is, making applications and desktops delivered from a central location perform with the same responsiveness and fidelity as if they were installed on a local PC, in the usual way.

Historically, this user experience has been one of the stumbling blocks for desktop virtualization in general. Older forms of Citrix Presentation Server (now rebadged and modernized under the XenApp label) and initial virtual desktop infrastructure (VDI) implementations very much tried to simplify management and otherwise deliver direct benefits for IT operations. Whether users liked using the products was secondary.

As a result, desktop virtualization has been mostly something used by what are often called "task workers." Think call centers and other groups of users with specific jobs to do and not much say about the tools they use to do it. In general, desktop virtualization promoters have focused too much on delivering benefits to IT and not enough on delivering benefits to users. (They've also arguably paid too little attention to keeping up-front costs down and relied too much on promises of soft cost savings down the road.)

One of the technology pieces that VMware is leaning on to improve user experience is the PC over Internet Protocol (PCoIP). PCoIP was originally developed by Teradici to improve the responsiveness and display quality of virtual desktops. However, in Teradici's initial implementation, specialized hardware was needed on both ends of the wire. This effectively made it a premium solution for situations in which cost wasn't a factor, such as for financial traders and government agencies for which security considerations are paramount.

VMware has worked with Teradici to create a software-only version of the protocol. Desktop virtualization Chief Technology Officer Scott Davis goes into a lot of the details on his blog.

It's a User Datagram Protocol-based server-side protocol that transmits compressed bitmaps or frames to the remote client. This has the advantage of being able to make real-time adjustments to account for the available bandwidth and latency of the communications channel; the display quality degrades, if there isn't enough bandwidth but things still "work."

Although details differ, there are similarities to Sun's Appliance Link Protocol--which is well-regarded for its ability to deal with poor-quality connections. (A downside of server-side protocols is that they consume processing horsepower on the server, where it tends to be more expensive, rather than on the client.)

VMware will continue to support other remote display protocols, most notably Microsoft's Remote Desktop Protocol. However, VMware is clearly positioning PCoIP as its favored technology and a point of competitive differentiation for VMware View in general.

Also in the graphics area, View 4 adds "multimonitor, adaptive display support--resolution optimization for each monitor, with an option to pivot and rotate the display output, supporting rich audio and video content with increased performance."

Other user experience enhancements generally relate to better integration with the overall desktop environment. For example, View Printing automatically discovers local printers without the need to install print drivers. View Limited Access provides a single point of authentication across VMware View environments, Windows Terminal Servers, Blade PCs, and remote physical PCs.

VMware View 4 comes in two editions. The Enterprise Edition includes the basics: VSphere 4 (the back-end server virtualization product), VCenter 4 (management), and View Manager 4 (for provisioning user access). It's priced at $150 per concurrent connection.

The $250-per-concurrent-user Premier Edition adds ThinApp 4 (for delivering ad hoc applications that aren't part of a master image) and View Composer (for managing images), both capabilities that would typically be desired in a large or sophisticated deployment.

VMware as a whole approaches the world from the perspective of the enterprise data center. Delivering desktops from that data center was somewhat of a sideshow. Is it now as focused on application delivery as, say, Citrix? Not really. But that said, desktop virtualization has moved beyond the sideshow stage at VMware.

Intel's James Reinders is an expert on parallelism; his most recent book covered the C++ extensions for parallelism provided by Intel Threaded Building Blocks. He's also the Director of Marketing and Business for the company's Software Development Products. In Part 1 of our discussion at the Intel Developers Forum in September we talked about how to think about performance in a parallel programming environment, why such environments give developers headaches, and what can be done about it.

Here, in Part 2, we move on to cloud computing, functional and dynamic languages, and what needs to happen with computer science education.

Few wide-ranging conversations these days would be complete without at least a nod to cloud computing which Reinders views as very much connected to the matter of parallel programming.

Cloud computing is parallel programming. You're solving the same problem. In fact, someone that's good at decomposing a program to run in parallel on a multicore or on a supercomputer... the same thought process is necessary to decompose a problem in cloud computing. What's different in cloud computing is that the cost of a connection or a communication between two different clouds is so high. You really need to get it right. It works best when a little message is sent, does an enormous amount of computing, and gets a little message back.

Data parallelism tends to be very fine-grained.

Task parallelism like we see with Cilk and Threaded Building Blocks is a little bit more coarse.

Cloud computing has to be very very coarse-grained parallelism.

But there's something common about how you have to think about it.

The tools that will let people do cloud computing, express a problem in cloud computing, may eventually just map onto a multicore.

The granularity that Reinders discusses refers to how small a chunk of computing can be, given the cost and latency of communications. Within a single processor, communications bandwidth is high and latencies low, so software can afford to perform a relatively small task and then synchronize the results. (Although moving large amounts of data can still be relatively "expensive" which is why data parallelism can be finer-grained than task parallelism; see Part 1 for further background on data parallelism.)

By contrast, external communication networks have limited bandwidth and are relatively slow--on the order of four or five orders of magnitude slower than communications within a system. Therefore, tasks have to be parceled out in relatively large chunks that, ideally, don't have to be packaged up with a significant amount of local data.

Next up was education. Here, Reinders' basic message was focusing on the theory before diving into the implementation details. I suspect that this highlights one of the key challenges: Parallel programming tends to require a solid grasp of programming theory and doesn't lend itself particularly well to just "hacking around" in the absence of that grounding.

I've been doing a lot in the area of teaching parallelism. What a lot of people think of right away is teach them locks, teach them mutexes [algorithms to prevent the simultaneous use of a common resource], teach about how to create a thread, destroy a thread. That's all wrong. You want to be talking at a higher level. How do you decompose an algorithm? What is synchronization in general? Why does it exist?

Things I would hope undergraduates would learn are parsing theory, DAG representations [a tool used to represent common subexpressions in an optimizing compiler], database schemas, data structures, algorithms. All these are high level, not things like [the programming language] Java. Parallel programming's like that too. You get hands-on touching the synchronization method or whatever but you want to teach the higher level key concepts.

Some people it's going to be more in-tune with their thinking but you try and teach it to everyone.

Given that most of today's languages weren't expressly designed for parallel programming, discussions about parallelism often turn to new programming languages. This means functional languages most of all but can also involve dynamic or scripting languages which generally handle more low-level details under the covers than do Java or C++.

Functional languages don't lend themselves to easy, or easily comprehensible, description. A common shorthand is that "Functional programming is a style of programming that emphasizes the evaluation of expressions, rather than execution of commands." But that probably doesn't help much if you don't already know what it is. As for Wikipedia's entry, Tim Bray--no programming slouch--called it fairly impenetrable. (Perhaps you begin to see the problem.)

A couple of things I'm interested in functionals for. We don't wake up one day and everyone uses. It's sequential semantics again and sequential semantics appeal to people and functional languages don't have them. But some people eat them up.

And they solve amazing problems. You can code things up in them that are much easier to understand than if they are written in a traditional language although they can be cryptic or terse to a lot of programmers.

Erlang [a functional language] has gotten a bit more and more usage. Maybe it is creeping in. It's not going to take over the world overnight but it seems like the one that might stay around. May be talking about it 20 years from now and saying, yeah, Erlang's been around for 25 years. It might be accepted as a language. It may have legs.

But even Java. [Unlike Erlang,] It appealed to people who programmed in C and C++; it didn't challenge them to think differently. And because of the strict typing and stuff it helps [the enterprise developer] to deploy certain types of apps.

Python [a dynamic language] is interesting. It is so popular with a lot of scientists. It's on my short list of things, where if we can figure out where to partner or extend some of the things we're doing, Python's on my short list of languages that we want to help with parallelism. Maybe some of our Ct technology would apply there. We'll see if other people agree with us. Think the concepts we're talking about are pretty portable. 

Finally, we concluded our discussion with hardware.Are there opportunities at the hardware and firmware level with memory subsystems or with specific technologies such as transactional memory? Sun Microsystems was very interested in transactional memory in the context of its now canceled "Rock" microprocessor. The basic concept behind transactional memory is to provide an alternative to lock-based synchronization by handling concurrency problems as they occur at a low-level rather than having the programmer protect against them all the time.

The best solutions tend to not be silver bullets so much as incremental. Nehalem [Intel's latest microprocessor generation] in a way probably helped us more than  anything in recent memory because we moved to the QuikPath interconnect and moved bandwidths up and latencies down. Larrabee [a many-core Intel microprocessor still under development] may pave the way with some innovations in interconnects. I think there may be some refinements needed. Interconnecting the processors is a classic supercomputer issue.

Transactional memory has slammed up against a very tough reality which is that hardware always wants to be finite; software solutions wants to be infinite. Think there's something there.I think the people looking at transactional memory have started to make observations about locks that may end up being useful. It's funny. The mission of transactional memory is to get rid of locks but the more they looked at it the more they understood about how locks behave. There might actually be possibilities to make locks behave better in hardware.

Can we do the hardware a little differently? Not the sexiest thing in the world. But as we move from single-threaded to  multi-threaded what complications are we creating things [that the hardware can help with]?

Even if you don't subscribe to the more extreme views of programming and software being in a crisis because of the move to multi-core, we're clearly in a transition. New tools are needed and programmers will have to adapt as well, to at least some degree.

Multicore processors are here to stay and the number of  cores that we'll see packed onto a single chip is only going to increase. That's because Moore's Law is only indirectly about performance; it's directly about increasing the number of transistors. And, for a variety of reasons, turning those transistors into performance today largely depends on cranking up the core count.

There's a downside to this approach though. Programs that consist of a single thread of instructions can only run on a single core. This in turn means that they're not going to get much faster no matter how many cores a chip adds. Running faster means going multi-threaded--splitting up the task and working on the different pieces in parallel. The problem is that programming multi-threaded applications introduces complications that don't exist with single-threading.

These complications and ways to overcome them was the topic of my conversation with James Reinders at the Intel Developers Forum in September. Reinders is the director of marketing and business for Intel's Software Development Products. He's an expert on parallelism and his most recent book covered the C++ extensions for parallelism provided by Intel Threaded Building Blocks.

In part 1 of this discussion we talked about how to think about performance in a parallel programming environment, why such environments give developers headaches, and what can be done about it.

Reinders began by noting that developers fall into roughly two groups when it comes to parallel programming: those who are still concerned about ultimate performance even in a parallel world and those who are just looking for a way to deal with it at all.

The challenge is understanding what we're trying to introduce, how to use parallelism, but with programmer efficiency. Because programmers don't need yet another thing to worry about. There's plenty of those out there.

And we need to be a little more relaxed about the performance. The people who start asking me about efficiency in every last cycle used and such--I characterize them as people we need to talk to more about our high-performance computing-oriented tools that give you full control. And other people are "I don't even know how to approach parallelism." I think there is a different set of ways to talk about the problem.

The problems with this second group comes down to the fact that most programmers are used to dealing with something called "sequential semantics." A detailed description of programming semantics is a complex computer science topic but, at a high level, sequential semantics means more or less what it sounds like it sounds; instructions follow one after another and execute in the order that they are written.

If you store the number "1" in variable A, then store the number "2" in variable B, and then add them together in a third instruction, you can be confident that the answer will be "3." It won't depend on timing vagaries that might have caused the addition to happen before the stores. Most people start out programming sequentially using languages designed for that purpose.

Parallel programming, on the other hand, introduces concepts like data races (the answer is dependent on the timing of other events) and deadlocks (in which two threads are each waiting for the other to complete so that neither ever does). Here's Reinders:

If you've ever managed and got a bunch of people working on a project together, one of the headaches you get is coordinating with each other. What did Fred say to Sally? They're doing things out of order or whatever. Parallel programming can give you that same sort of headache.

The programming terminology you'll hear the compiler people use is "sequential semantics." One of the interesting areas is what can we do if we ensure sequential semantics. We recently acquired a team in Massachusetts who were working for a company called Cilk Arts.

Our hope is that Cilk can do a subset of what Threaded Building Blocks [TBB] can but preserve sequential semantics. We think we can do sequential semantics, do a subset of what TBB does, since we're introducing keywords into the compiler--that has some disadvantages because it's not as portable--but we think we might be able to magically give you sequential semantics and not give up performance. That's a big if.

Now why would we invest in that?

Because there are a lot programmers who have been getting along just fine with sequential programming. But when you tell them to add this or that for parallelism, a big thing that trips them up is that you no longer obey sequential semantics; you have more than one thing running around and you get data races, deadlocks, and it doesn't feel comfortable.

Now some people will argue that you need to do these things to get good performance. We have the feeling that in some cases you don't need to take that big of a leap to get pretty good performance.

And no one's going to criticize your app on a quad core for being only 70 percent efficient.

From there we moved on to data parallelism which focuses on distributing data across processing elements. It contrasts with the task parallelism that we commonly associate with the term parallel programming. Pervasive DataRush is one commercial product based on a data parallelism model. APL, the language with the strange symbols (for those with long memories), is often considered the first data parallel language. There have been a variety of others, often extensions to more conventional languages like C and FORTRAN, but none were widely used.

The other thing we're looking at is data parallelism. And that's where we acquired the RapidMind team and combined them with our Ct [C for Throughput Computing] team.

Data parallelism just takes it one step further. Data parallelism is all about the parallelism in the data. So you're talking about the data when you program.

And once you start talking about the data, the tools underneath can move the data around. Leaving the data management up to the programmer [as with Cilk and TBB] turns out to be a terrific headache. This applies equally to a cluster where they don't share memory or a GPU and a CPU in the same system.

But a language like RapidMind or Ct can address that problem. And CUDA and OpenCL can too [frameworks primarily oriented towards heterogeneous processing that uses graphics cores for computing tasks] but RapidMind and Ct are at a much higher level of abstraction which means that we're betting on the idea that we can attract more developers and give up some efficiency.

Part 2 of our conversation will cover cloud computing, functional and dynamic languages, and what needs to happen with respect to programmer education.

Correction at 7:15 a.m. PST November 4: At one time, Red Hat had planned to ship an embedded KVM hypervisor based on Fedora. But the Red Hat Enterprise Virtualization Hypervisor uses the RHEL 5.4 kernel and thereby picks up the same hardware verification portfolio.

With Tuesday's release of Red Hat Enterprise Virtualization Hypervisor and Red Hat Enterprise Virtualization Manager (RHEV-M) for servers, the company has completed the first phase of a server virtualization rollout that effectively now puts KVM front and center. Red Hat released KVM commercially for the first time in September as part of Red Hat Enterprise Linux 5.4.

KVM is a server virtualization technology that Red Hat acquired when it bought Qumranet in 2008. Red Hat favors KVM over the other primary open-source hypervisor, Xen, for both business and technical reasons. (Although, as of version 5.4, Xen remains the default hypervisor for RHEL.)

The business reason is that, while Red Hat has made contributions to Xen, competitors are far more associated with the project. Novell, the owners of the only other major enterprise Linux distribution, ran especially hard with Xen early on. And Citrix, not a direct competitor but certainly a major virtualization player, bought XenSource, the commercial entity formed by Xen's creators.

From a technical perspective, Red Hat's issue is that it's hard to keep Xen and the Linux kernel in sync. Xen's a standalone hypervisor layer but it has deeply invasive hooks into the Linux kernel and, therefore, keeping the two working together takes a lot of development and testing effort. It's a bit reminiscent of how new versions of the Veritas file system had to be carefully matched to new versions of Solaris or HP-UX.

By contrast, KVM is kernel-based. This means that it is actually part and parcel of the Linux kernel rather than a quasi-independent piece of software. In part for this reason, it's KVM that is now included in the mainline Linux kernel as of version 2.6.20.

As of version 5.4, an instance of RHEL can host guest virtual machines running RHEL 5 and other operating systems including Windows Server 2008. This announcement adds Red Hat Enterprise Virtualization Hypervisor, something that is often referred to as an "embedded hypervisor." It uses the same RHEL 5.4 kernel as Red Hat's full enterprise distribution.

Embedded hypervisors have taken off more slowly than many of us expected. But all the major virtualization players offer one so Red Hat needed to as well.

From my perspective, the Red Hat Virtualization Manager is more significant. On the one hand, management is important to--indeed central to--virtualization. On the other hand, it's an area where Red Hat has lagged. CTO Brian Stevens admitted as much to me when we spoke at the company's financial analyst day last month when he said that RHEV-M "has been a huge missing ingredient."

Red Hat historically mostly focused on updating packages. This is a reflection of the broader Linux and open-source ecosystem in general. Projects like Nagios and, more recently, GroundWork notwithstanding, management doesn't play well to the strengths of open source because it's such a "high surface area" application. But Red Hat had to attack management from some angle unless it was prepared to just cede that area of differentiation and potential point of control to system makers and others.

RHEV-M is Red Hat's first step toward remedying this deficiency. It seems a necessary move especially given that KVM is likely to be used, at least initially, as part of a Red Hat software stack and therefore Red Hat pretty much has to support the tools to manage KVM if it's to gain any market traction.

That said, this is very much a first step. The initial product only manages KVM. Furthermore, the management server has to be running Windows Server 2003 which you would rightly think a rather odd decision from a company that is one of the pioneers of open source. (Apparently, this was a decision by Qumranet and Red Hat has not yet developed a version that can run on Linux.)

Red Hat has clearly prioritized getting a usable if limited product into customers' hands. They trotted out one such at their financial analyst day. Dave Costakos of Qualcomm was happy with what he saw. He told me that they wanted a Web-based interface, which RHEV Manager has. He also liked the integration with Active Directory and other directory systems, the role-based access controls, and the provisioning capabilities.

Overall, Red Hat's virtualization play remains less filled in than do the plays of others. But it's now started in a systematic way.

Over the years, we've seen a variety of approaches intended to meld multiple small servers into a single larger system. 3Leaf Systems is the latest. On November 3, it introduced a Dynamic Data Center Server (DDC-Server) for AMD Opteron processors. The DDC-Server combines a custom DDC-ASIC chip with software to create a symmetric multiprocessing server with 32 6-core AMD "Istanbul" processors and 1 terabyte of memory.

The system, together with the InfiniBand switch required to interconnect the  server components, 8TB of storage, and 3Leaf's software, is priced at $250,000. A smaller $99,000 version is also available. However, these systems should be thought of primarily as proof of concepts intended to create proof points with customers and to provide system makers with a tangible product. 3Leaf's go-to-market plan is to sign up system original equipment manufacturers and sell them ASIC (application-specific integrated circuit) chips and software--not to itself be a seller of systems.

The basic concept behind 3Leaf's design has quite a few antecedents.

In the 1990s, Data General and Sequent came up with large Unix server designs that connected "standard high volume" (SHV) x86 modules with cables using a protocol from Dolphin Technology called SCI. The component modules were never as standard or high volume as the SHV term implied but the approach still reduced development costs and increased the flexibility of the system relative to the more monolithic designs that characterized most large SMP servers of the day.

More recently, Virtual Iron developed a distributed hypervisor that could not only subdivide a single server in the vein of server virtualization products like VMware's ESX Server, but could also meld multiple smaller systems into large ones on the fly. (Virtual Iron later abandoned its proprietary hypervisor in favor of Xen and was later absorbed by Oracle.)

ScaleMP's vSMP Foundation is the current product for aggregating x86 servers that is probably most comparable to 3Leaf's. To date, it's been primarily focused on high-performance computing. The key distinction is that, unlike ScaleMP, 3Leaf uses a custom ASIC in addition to software. Both companies are primarily focused on InfiniBand as their interconnect although there is nothing architectural to prohibit the use of 10-Gigabit Ethernet over time. From a technical perspective, 3Leaf is essentially layering its own coherency protocol on top of InfiniBand. The current product uses the same socket as the AMD processor. However, 3Leaf also has a license for Intel's QuickPath Interconnect.

3Leaf says that, by developing an ASIC that gets into coherent memory transactions at the cache level, they are able to get better performance across a wider range of workloads than a purely software-based approach can.

Performance has been a stumbling block with this approach historically.

An SMP server, however constructed, is characterized by the fact that it is a shared memory architecture. This means that any processor can directly access any memory in the system. In general, this makes for a simpler programming model than distributed memory architectures, such as clusters, in which a lot of the work associated with making sure you're working with latest data is shifted from hardware to software.

How quickly a given processor can get to the memory that it needs plays a big part in a system's performance. In fact, for some workloads such as database transaction processing, memory access times can be the single factor that most affects how fast a system is. As a result, traditional large server designs incorporated expensive hardware such as crossbars to keep memory traffics flowing across the entire system quickly.

Today's small servers have equally speedy and high-bandwidth memory links--indeed their compact footprint can help to reduce latency even further. However, once you combine multiple nodes, the time it takes for a processor to access memory on another node can rise dramatically. The exact numbers depend on many factors, including what else is going on in the system at the time. But, as a rule of thumb, it takes at least twice as long to access memory on another node than if it were local--and could take several multiples of that. In other words, memory access is non-uniform; NUMA is the term often used.

Over time, operating systems have gotten much better at keeping processing and associated memory physically close to each other. Certain workloads are also less sensitive to NUMA designs than others. Many HPC, analytics, and business intelligence applications involve fewer of the sort of memory updates that tend to drag down performance in NUMA architectures than does typical enterprise online transaction processing.

It's also the case that, today, large SMP is as much about having a large and flexible pool of hardware resources for server virtualization as it is about having a single large SMP image. Thus, in many respects, large SMP is increasingly about management rather than monolithic application performance. Which is one of the reasons that we're seeing a general trend towards modularity in all SMP designs.

Thus, the SHV approach to SMP system design arguably sits closer to the mainstream than it has in the past.

As 3Leaf's Shahin Khan told me, the key factors with this approach are it "had better be low cost and work." Performance has to be acceptable over at least an interesting subset of workloads and there can't be a significant price premium over the constituent systems and hardware. And ultimately, for 3Leaf, success will result from convincing one or more major system OEMs that the time has arrived to add a system or systems based on this approach.

Last week's virtual version of the roaming CloudCamp conference was a good opportunity to check the pulse of cloud computing's evolution.

It struck me that we continue to see two very different groups of attendees at events such as this.

One is the "clouderati," the vendors involved with cloud computing in some form or another, and sophisticated users who grok cloud computing and its implications for their organization. This crowd is so past definitional debates and analogies to the electrical grid. They just want to get on with specific issues--such as dealing with audit requirements in a world where you increasingly can't just walk into a datacenter and point to the physical server where an application is running.

The other group is still a bit fuzzy on the general concept. Does cloud computing just mean Amazon Web Services? Where does software as a service fit? Is it just a load of hype? Is it safe?

It's not hard to understand why there is a fair bit of confusion. Cloud computing has become a sort of blanket term for where computing is going. Think of it as a synonym for "computing.next." It represents a shift to an operational model in which applications don't live out their lives on a specific piece of hardware and in which resources are more flexibly deployed than was the historical norm.

Cloud computing is therefore not a single technology or even a single approach but rather a collection of technologies and approaches that collectively represent the direction that computing is headed. I see nothing wrong with this. Many of the benefits espoused for these new approaches to computing are genuine. To the degree that "cloud computing" offers a convenient rallying point to get users headed there, that seems for the good.

But specific things are easier to grapple with than general paradigms. And cloud computing started life as something fairly narrow as articulated in author Nick Carr's The Big Switch. (The irony is that, not only is the cloud computing concept bigger than the Big Switch concept of a few, huge mega-service providers, it now seems unlikely that the degree of centralization and fundamental change in economic model envisioned by Carr will happen any time soon.) Whereas today, we see cloud computing used sometimes to mean computing.next and sometimes to mean a specific technology approach that someone is either promoting or denigrating.

Cloud computing incorporates and makes use of many individual sharply-defined techs of course. But, increasingly, think of the broad term as applying to a way of thinking about computing rather than the specifics of how it's done.