The Pervasive Data Center

The nice thing about standards is that there are so many of them.

This old saw is arguably less true than in years past. Today, for a lot of reasons, there's more pressure to reach agreement on one way to do a certain thing. (Think the HD DVD vs. Blu-ray debacle for an example of what happens when vendors can't agree on a single approach.)

Standards aren't a single thing. Some have been blessed with the appropriate incantations by some official or quasi-official body. Others come from an industry consortium. And still others are "de facto" (or at least began life that way)--the result of a dominant company or just a default way of doing things.

USB Flash Drive

(Credit: Ambuj Saxena, Flickr (under CC))

The purist will argue that just being widely used doesn't make something a standard. I agree up to a point and only use the "standard" term in this case for things for are truly ubiquitous. Contrariwise, a rigorous formal ratification process is no guarantee of success.

But some standards do win big and become part of just how IT gets done. Here are some of them.

Like many other successful standards, Ethernet has remained a fixture in local area networks for so many years in part by adapting to many successive waves of technology. First developed in the famous Xerox PARC labs in the mid-1970s, it initially ran over coaxial cable but soon moved to twisted pair cable with the 10 Mbit/second generation. 10 Gbit/second Ethernet is now starting to roll out along with a variety of additions to the specification that make it more suitable as a high-performance unified fabric.

Ethernet's initial success resulted in no small part from coordinated standardization efforts beginning in the IEEE. This helped it beat out alternatives, most notably IBM's Token Ring. Over time, Ethernet's ubiquity and the cost benefits provided by this volume helped it largely stave off server interconnect challengers. InfiniBand has had wins in high-performance computing and certain other clustering applications, but it didn't displace Ethernet as a "server area network" as early promoters had hoped.

PCI, Peripheral Component Interconnect, had its beginnings as an Intel-developed bus for connecting internal cards within systems. The version 1.0 spec came out in 1992. Given the ubiquity of PCI these days, it's easy to forget that it only replaced a plethora of other busses both standardized and proprietary in x86 and, later, large Unix servers based on other processors over the course of nearly a decade.

Nor was the process steady. Although PCI was initially introduced in part to replace the VESA Local Bus for graphics cards--which it eventually did--PCI was itself replaced by AGP (Accelerated Graphics Port) for a time prior to the PCI Express generation.

PCI Express makes for an interesting case study in the marketing of standards. With technology bumping up against the limits of parallel I/O busses like conventional PCI, the Arapahoe Working Group--spearheaded by Intel--started pushing a new serial interconnect standard in about 2001.  Arapahoe's success was by no means pre-ordained. AMD's HyperTransport was one alternative among several.

Arapahoe required hardware that was largely different from PCI but it was compatible with PCI's software model in a number of respects. And this was enough to get Arapahoe adopted by the keeper of the PCI standard, the PCI-SIG, and get the SIG's imprimatur on what would now be called PCI Express. And that helped make it the obvious heir to PCI. Names matter. (Here's a more detailed accounting of PCI Express and its history.)

It's easy to forget just how painful it could be, in the years before USB (Universal Serial Bus), to connect external peripherals to a computer system. RS-232, a long-used and successful standard in its own right, was the most common way. It was also a way that could easily devolve into examinations of cable pin-outs, interrupt channels, and memory addresses.

USB was a cooperative effort by a group of large technology vendors who founded a non-profit corporation to manage the specification. Version 1.0 was introduced in 1996. Now up to version 3.0, USB has become the standard way to connect external peripherals to PCs; it's also commonly used on servers for devices such as printers.

There's a spec for wireless USB but, like other standards intended to connect peripherals to computers wirelessly, it's never taken off. The current such "personal area network" getting the most buzz is My WiFi from Intel.

USB's primary competition has been FireWire, Apple's name for IEEE 1394. Unlike USB, it does not need a host computer and is faster than the USB 2.0 generation. However, it didn't catch on widely in the computer industry outside of Apple (which is phasing it out in favor of USB) and video equipment.

TCP/IP refers to the combination of two protocols: Transmission Control Protocol and Internet Protocol. Together, they are among the most important pieces of software underpinning the Internet which transitioned to using TCP/IP in 1983. Work on TCP began under the auspices of the Defense Advanced Research Projects Agency (DARPA) a decade earlier but, along the way, the software stack was re-architected to add IP as the early Internet grew.

Like many of the Internet's building blocks, TCP/IP was firmly entrenched before commercial interests got involved to any significant degree and, indeed, before most of the world at large had any real notion of the Internet's existence. The general public came to know the Internet through the World Wide Web, an outgrowth of Tim Berners-Lee's development of HTML at CERN, in the 1990s. Thus HTML, as well, is a key standard.

At the time that TCP/IP was gaining momentum, the International Organization for Standardization (ISO) spearheaded a large project to standardize networking. The "OSI model" remains the standard way to think about layers of the networking stack. If you talk about a switch being "Layer 4," you're using OSI terminology. But the specific protocols developed to go with the model were never widely used. (TCP/IP largely maps to the layers defined in the OSI model.)

The x86 architecture is perhaps the canonical example of a de facto standard driven primarily by a single vendor: Intel. Microsoft Windows is also in the running, but it was very arguably x86's ubiquity in a segment of the market open to relatively low-cost packaged software that made the rise of Windows possible. Over the past decade, AMD has also driven x86 innovations--most notably 64-bit extensions. However, it was Intel that had the biggest hand in shifting the industry from a structure in which each company did everything from fabricating processors to writing operating systems to developing databases to one in which different companies tend to specialize in one part of the technology ecosystem.

x86 emerged as a dominant chip architecture for a variety of reasons. IBM designed Intel's 8088 into the first important business PC. It got this win and others at a time when the world was rapidly computerizing. And Intel optimized itself to ride key technology trends while divesting itself of businesses, such as memory, as they commoditized.

Finally, here are a few others that could make a list like this one:

Wi-Fi played a big role in making personal computers more mobile--which is why Intel pushed it so hard.

VGA is the computing video standard that finally helped merge a rather splintered landscape and had a good long reign. (The latest video interconnect trend is a shift to HDMI--representing a coming together of computing and consumer electronics standards.)

SCSI was the first storage interconnect to merge in a big way a disparate set of existing connection schemes, both proprietary and more or less standardized. However, storage has remained an area where different standards are used for different purposes. That's changing to a degree with SATA, however, which we now see in both PCs and data centers.

This is the time of year to take stock in where high-performance computing (HPC) sits and where it is headed. That's because the SC09 conference is taking place in Portland, Ore., this week and it's the biggest HPC conference around.

SC is an odd duck as conferences go. Last year it had more than 10,000 attendees and, yet, it's a largely volunteer-organized event in a world where trade shows of this scope are packaged by conference specialists or some specific corporation. Think the much-renamed LinuxWorld  (run by IDG) or VMworld (run by VMware).

"SC" comes from supercomputing. Today's large computer complexes are typically not supercomputers in the sense of a specialized architecture only suitable for a specific type of technical computing. Rather, as Ashlee Vance notes in The New York Times, "The supercomputing world was long dominated by systems that required specialized chips, memory systems and networking technology. But about 10 years ago, researchers realized they could link thousands of cheaper machines running on mainstream chips and achieve pretty solid performance."

Thus an HPC event is no longer about supercomputers per se (although the term is still used as a convenient moniker for a collection of resources managed as a single entity in a single location). Rather it's about the computing components, the interconnects, the storage, and the software that ties everything together and the applications that run on top.

The Top500 nicely illustrates the evolution of HPC over time. This list, released twice annually, ranks the largest publicly acknowledged supercomputers--as the term is used today--on the basis of a somewhat simplistic, but objective, benchmark. The Top500 entries are certainly not typical of mainstream HPC; they're the biggest of the big. But they nonetheless provide some quantitative insight into important trends.

The newest iteration of the list was released Friday. There were no striking departures from the trends of the last few years, but there was some continued evolution that's worth taking note of.

The continued rise of InfiniBand. InfiniBand is a system interconnect that offers a higher performance alterative to the ubiquitous Ethernet. Although its initial backers envisioned a broader role for the technology, it's settled nicely into HPC and, to a lesser degree, back-end commercial data center functions like database clusters where low latency and high bandwidth are also paramount. (The Sun/Oracle Exadata appliance uses InfiniBand for example.)

(Credit: TOP500.org)

InfiniBand's initial growth in HPC wasn't so much about displacing Ethernet as it was displacing the fractured collection of high-performance interconnects that preceded it. Myricom's Myrinet and Quadrics' QsNet were the most common of these, but there were many. This year InfiniBand is deployed on 181 of the Top500, a 28 percent increase from a year ago.

That's a striking increase clearly. But what is perhaps more striking is that about half that increase came at the expense of Ethernet rather than mopping up a variety of older or proprietary connection technologies. This shift started between 2007 and 2008 but was even more pronounced this year.

It's certainly possible that the next 10GbE generation of Ethernet, which today is essentially absent from the list, could again push Ethernet's numbers higher. However, whatever the specific technology, the message that I take away is that large computer clusters are starting to favor more optimized interconnects even if they and the components they connect are largely off-the-shelf.

And we see an analogous trend with the proliferation of blade servers as well. Blades, a more modular and pluggable approach to system design, have proven popular in many enterprises and midmarket companies, in part, because they help bring together computing, storage, and networking technologies into a single integrated whole. That type of integration isn't of much interest in HPC. Rather, blades play to HPC by offering high densities and reducing cable count and complexity.

In fact, among x86 servers at any rate, dominance is not too strong a word to describe the presence blades in the Top500. Consider just one vendor, Hewlett-Packard. HP has 208 ProLiant systems on the list. A full 203 of these, almost 98 percent, are ProLiant c-Class blades.

Collectively, these trends suggest what might be thought of as a trend toward building optimization around standardization. In the main, especially as one moves down from the very top of the list, the Top500 is composed mostly of systems using mainstream technologies such as x86, Linux, and standard interconnects. Clusters are the dominant architecture.

But we're increasingly not seeing mere rackmount servers connected by Gigabit Ethernet. As the systems on the Top500 list grow in capability, we're seeing more focus on how they're packaged, powered, and connected.

On the one hand, vendor analyst events are a good opportunity to spend focused time diving deep into individual products, roadmaps, and corporate initiatives. On the other, they're a useful forum for getting the feel of a company's overall zeitgeist in a way that narrower discussions don't. EMC's event, held last week in Franklin, Mass., was no exception.

(Credit: EMC)

Perhaps the single thing that struck me most about the event as a whole was the full integration of VMware into the discussion as a whole. I've been following both companies since before EMC acquired VMware in 2003. In the years since, although there were the obligatory nods to joint development work and "better together," VMware aggressively maintained a distance that was hardly limited to the 3,000 miles between VMware's Palo Alto, Calif., headquarters and EMC in Massachusetts. VMware's presence at EMC analyst events was largely relegated to a few off-hand mentions and perhaps a desultory breakout session given by a junior marketing person.

This year couldn't have been more different. VMware was very much woven into just about every discussion and one of VMware's senior technologists shared a panel with representatives from EMC and Cisco Systems. One thing that has changed, of course, is the ouster of VMware founder and CEO Diane Greene in 2008. It was Greene who most vocally kept EMC at arm's length. It's also the case that virtualization is increasingly at the center of everything that EMC does, so how could VMware not be an integral part?

This pervasive virtualization theme carried through to EMC VP Jon Peirce's discussion about EMC's internal IT infrastructure as well. EMC IT is using VMware to virtualize as much as possible. This includes doing database testing on a Cisco Unified Computing System (UCS) in advance of a planned migration off Sun E25000 UltraSPARC-based servers.

An initial Virtual Desktop Infrastructure (VDI) deployment also uses UCS in the form of a vBlock--a preconfigured package that combines products from Cisco, EMC, and VMware. EMC has about 200 users on VDI today and expects to roll out to several thousand next year starting in their Franklin facility. VDI and associated forms of desktop virtualization are a favorite technology of CEO Joe Tucci, who would like to move toward a platform-agnostic client strategy.

The ultimate goal is what sometimes goes by BYOPC (Bring Your Own PC), in which employees provide their own notebook computers, perhaps purchased with the help of a stipend. Even today, many of the EMC execs at the event were sporting Macs, even though IT doesn't officially support them.

Another hot topic at the event was multi-tier storage, in this case automatic storage tiering that intelligently moves data between Flash-based storage and conventional disk drives. EMC's technology here is called FAST and will roll out on Symmetrix V-Max arrays.

Flash drives can be much faster than SATA disks--or even high-performance Fibre Channel drives--but they're also much more expensive on a per-GB basis. The idea behind FAST is to automate the placement of data based on the way its accessed. For example, a database index that is frequently read and written to will migrate to high performance flash while older data that hasn't been touched for a while will move to slower, cheaper disks.

Disks being used to store rarely accessed archival data can even be deduped, compressed, and even spun down to reduce overall data center power consumption. Tape isn't part of this vision; Tucci opined that "Backup to and recovery from tape is dead."

The idea of storage tiering isn't new. Hierarchical storage management (HSM) has been around for well over a decade. However, in practice, it's mostly ended up being about moving old files to tape for archive purposes. (EMC itself has a product in this vein: Legato DiskXtended.) FAST is something more transparent and more dynamic.

There are analogs between FAST and the storage pooling that is part of Sun Microsystems' ZFS filesystem. EMC argues that the function belongs on the storage device rather than the server because the array is where data access from multiple systems and applications come together.

It's unsurprising that EMC wants storage to be at the center of things. This is a company, after all, whose tagline is "where information lives." It is, however, worth remembering that this is a different lens through which to view the world than system vendors tend to choose--and, for that matter, than VMware chose historically.

Correction at 7:15 a.m. PST November 4: At one time, Red Hat had planned to ship an embedded KVM hypervisor based on Fedora. But the Red Hat Enterprise Virtualization Hypervisor uses the RHEL 5.4 kernel and thereby picks up the same hardware verification portfolio.

With Tuesday's release of Red Hat Enterprise Virtualization Hypervisor and Red Hat Enterprise Virtualization Manager (RHEV-M) for servers, the company has completed the first phase of a server virtualization rollout that effectively now puts KVM front and center. Red Hat released KVM commercially for the first time in September as part of Red Hat Enterprise Linux 5.4.

KVM is a server virtualization technology that Red Hat acquired when it bought Qumranet in 2008. Red Hat favors KVM over the other primary open-source hypervisor, Xen, for both business and technical reasons. (Although, as of version 5.4, Xen remains the default hypervisor for RHEL.)

The business reason is that, while Red Hat has made contributions to Xen, competitors are far more associated with the project. Novell, the owners of the only other major enterprise Linux distribution, ran especially hard with Xen early on. And Citrix, not a direct competitor but certainly a major virtualization player, bought XenSource, the commercial entity formed by Xen's creators.

From a technical perspective, Red Hat's issue is that it's hard to keep Xen and the Linux kernel in sync. Xen's a standalone hypervisor layer but it has deeply invasive hooks into the Linux kernel and, therefore, keeping the two working together takes a lot of development and testing effort. It's a bit reminiscent of how new versions of the Veritas file system had to be carefully matched to new versions of Solaris or HP-UX.

By contrast, KVM is kernel-based. This means that it is actually part and parcel of the Linux kernel rather than a quasi-independent piece of software. In part for this reason, it's KVM that is now included in the mainline Linux kernel as of version 2.6.20.

As of version 5.4, an instance of RHEL can host guest virtual machines running RHEL 5 and other operating systems including Windows Server 2008. This announcement adds Red Hat Enterprise Virtualization Hypervisor, something that is often referred to as an "embedded hypervisor." It uses the same RHEL 5.4 kernel as Red Hat's full enterprise distribution.

Embedded hypervisors have taken off more slowly than many of us expected. But all the major virtualization players offer one so Red Hat needed to as well.

From my perspective, the Red Hat Virtualization Manager is more significant. On the one hand, management is important to--indeed central to--virtualization. On the other hand, it's an area where Red Hat has lagged. CTO Brian Stevens admitted as much to me when we spoke at the company's financial analyst day last month when he said that RHEV-M "has been a huge missing ingredient."

Red Hat historically mostly focused on updating packages. This is a reflection of the broader Linux and open-source ecosystem in general. Projects like Nagios and, more recently, GroundWork notwithstanding, management doesn't play well to the strengths of open source because it's such a "high surface area" application. But Red Hat had to attack management from some angle unless it was prepared to just cede that area of differentiation and potential point of control to system makers and others.

RHEV-M is Red Hat's first step toward remedying this deficiency. It seems a necessary move especially given that KVM is likely to be used, at least initially, as part of a Red Hat software stack and therefore Red Hat pretty much has to support the tools to manage KVM if it's to gain any market traction.

That said, this is very much a first step. The initial product only manages KVM. Furthermore, the management server has to be running Windows Server 2003 which you would rightly think a rather odd decision from a company that is one of the pioneers of open source. (Apparently, this was a decision by Qumranet and Red Hat has not yet developed a version that can run on Linux.)

Red Hat has clearly prioritized getting a usable if limited product into customers' hands. They trotted out one such at their financial analyst day. Dave Costakos of Qualcomm was happy with what he saw. He told me that they wanted a Web-based interface, which RHEV Manager has. He also liked the integration with Active Directory and other directory systems, the role-based access controls, and the provisioning capabilities.

Overall, Red Hat's virtualization play remains less filled in than do the plays of others. But it's now started in a systematic way.

Over the years, we've seen a variety of approaches intended to meld multiple small servers into a single larger system. 3Leaf Systems is the latest. On November 3, it introduced a Dynamic Data Center Server (DDC-Server) for AMD Opteron processors. The DDC-Server combines a custom DDC-ASIC chip with software to create a symmetric multiprocessing server with 32 6-core AMD "Istanbul" processors and 1 terabyte of memory.

The system, together with the InfiniBand switch required to interconnect the  server components, 8TB of storage, and 3Leaf's software, is priced at $250,000. A smaller $99,000 version is also available. However, these systems should be thought of primarily as proof of concepts intended to create proof points with customers and to provide system makers with a tangible product. 3Leaf's go-to-market plan is to sign up system original equipment manufacturers and sell them ASIC (application-specific integrated circuit) chips and software--not to itself be a seller of systems.

The basic concept behind 3Leaf's design has quite a few antecedents.

In the 1990s, Data General and Sequent came up with large Unix server designs that connected "standard high volume" (SHV) x86 modules with cables using a protocol from Dolphin Technology called SCI. The component modules were never as standard or high volume as the SHV term implied but the approach still reduced development costs and increased the flexibility of the system relative to the more monolithic designs that characterized most large SMP servers of the day.

More recently, Virtual Iron developed a distributed hypervisor that could not only subdivide a single server in the vein of server virtualization products like VMware's ESX Server, but could also meld multiple smaller systems into large ones on the fly. (Virtual Iron later abandoned its proprietary hypervisor in favor of Xen and was later absorbed by Oracle.)

ScaleMP's vSMP Foundation is the current product for aggregating x86 servers that is probably most comparable to 3Leaf's. To date, it's been primarily focused on high-performance computing. The key distinction is that, unlike ScaleMP, 3Leaf uses a custom ASIC in addition to software. Both companies are primarily focused on InfiniBand as their interconnect although there is nothing architectural to prohibit the use of 10-Gigabit Ethernet over time. From a technical perspective, 3Leaf is essentially layering its own coherency protocol on top of InfiniBand. The current product uses the same socket as the AMD processor. However, 3Leaf also has a license for Intel's QuickPath Interconnect.

3Leaf says that, by developing an ASIC that gets into coherent memory transactions at the cache level, they are able to get better performance across a wider range of workloads than a purely software-based approach can.

Performance has been a stumbling block with this approach historically.

An SMP server, however constructed, is characterized by the fact that it is a shared memory architecture. This means that any processor can directly access any memory in the system. In general, this makes for a simpler programming model than distributed memory architectures, such as clusters, in which a lot of the work associated with making sure you're working with latest data is shifted from hardware to software.

How quickly a given processor can get to the memory that it needs plays a big part in a system's performance. In fact, for some workloads such as database transaction processing, memory access times can be the single factor that most affects how fast a system is. As a result, traditional large server designs incorporated expensive hardware such as crossbars to keep memory traffics flowing across the entire system quickly.

Today's small servers have equally speedy and high-bandwidth memory links--indeed their compact footprint can help to reduce latency even further. However, once you combine multiple nodes, the time it takes for a processor to access memory on another node can rise dramatically. The exact numbers depend on many factors, including what else is going on in the system at the time. But, as a rule of thumb, it takes at least twice as long to access memory on another node than if it were local--and could take several multiples of that. In other words, memory access is non-uniform; NUMA is the term often used.

Over time, operating systems have gotten much better at keeping processing and associated memory physically close to each other. Certain workloads are also less sensitive to NUMA designs than others. Many HPC, analytics, and business intelligence applications involve fewer of the sort of memory updates that tend to drag down performance in NUMA architectures than does typical enterprise online transaction processing.

It's also the case that, today, large SMP is as much about having a large and flexible pool of hardware resources for server virtualization as it is about having a single large SMP image. Thus, in many respects, large SMP is increasingly about management rather than monolithic application performance. Which is one of the reasons that we're seeing a general trend towards modularity in all SMP designs.

Thus, the SHV approach to SMP system design arguably sits closer to the mainstream than it has in the past.

As 3Leaf's Shahin Khan told me, the key factors with this approach are it "had better be low cost and work." Performance has to be acceptable over at least an interesting subset of workloads and there can't be a significant price premium over the constituent systems and hardware. And ultimately, for 3Leaf, success will result from convincing one or more major system OEMs that the time has arrived to add a system or systems based on this approach.

It used to be that system management products were for the care and feeding of individual servers. They could deal with many of them, sure, and might even have had tools aimed at automating repetitive operations. But, fundamentally, they mostly looked at systems in isolation.

Enterprise management tools, on the other hand, looked at the IT infrastructure big picture. Sophisticated and complex, tools like CA's Unicenter, HP's OpenView, and IBM's Tivoli were the aggregation point for alerts and reports about the health of an organization's IT. But they rarely actually did anything; they watched for problems but it was other software or system administrators that had to actually swing into action.

The bottoms-up orientation of system management tended to win out over time. Enterprise management was never displaced--exactly. But it did long seem as if many of the products in the enterprise management space sat far from where the interesting action was in the data center.

However, today, we're seeing a shift to system management that happens at the level of the data center as a whole or at least a virtualized pool of systems and applications. Virtualization is one of the drivers here. Another is "private clouds" or, if you prefer, a more dynamic and services-oriented view of IT resources.

As a result, system management products are starting to take on more and more of the roles that were traditionally associated with enterprise management. We're also seeing systems management meet enterprise management in the middle, so to speak. IBM's VMControl announcement on October 20 is a case in point.

As IBM puts it in their release: "VMControl allows combinations of physical and virtual IBM servers to be managed as a single entity. This approach--known as system pooling--expands the benefits of virtualization by helping corporate data centers simplify complex management functions and better share and prioritize use of critical resources such as processing power, memory and storage."

The new product, IBM Systems Director VMControl Enterprise Edition, is focused on virtualized environments. It supports IBM's PowerVM and z/VM as well as x86 virtualization technologies such as VMWare, Hyper-V and open x86 virtualization solutions. IBM plans to first offer it on IBM Power Systems running AIX in December, 2009 with other platforms coming next year.

VMControl Enterprise Edition works in concert with Tivoli; IBM also announced "a new version of Tivoli Provisioning Manager that provides enhanced automation of the manual tasks of provisioning and configuring servers, operating systems, middleware, software applications, storage and network devices." As I've discussed previously, Tivoli is very much a central part of how IBM views cloud computing and therefore how it thinks about the evolution of the enterprise data center.

PowerVM itself, as its full name implies, is part of IBM's Systems Director family. This is IBM's systems management portfolio; rough counterparts are HP Systems Insight Manager, Dell OpenManage, and Sun xVM Ops Center. Systems Director has been the recipient of considerable development and marketing attention in recent years that have greatly improved its integration across IBM's disparate product lines as well as its overall functionality.

Virtualization is no longer just about server consolidation. It does that, sure, and thereby reduces the number of physical servers that an organization needs to purchase. But, especially in enterprises, it's increasingly as much about resource pools and services (such as disaster recovery) enabled by virtualization as it is about consolidation. And that makes the need for management more rather than less.

The debate over single-function server appliances versus general-purpose servers is a long-standing one.

Appliances first came onto the scene in the late 1990s during the first Internet boom. They focused on a particular task, such as Web serving, and were designed to be ready to install with minimum muss, fuss, or skill. This assembly line approach to server farms was to be the secret sauce that made possible infinite growth without infinite IT staff.

Cobalt Networks was perhaps the best known and most sophisticated of the companies to offer appliances. Sun Microsystems later acquired Cobalt and then failed to successfully integrate it. This arguably presaged the mixed history of subsequent Sun acquisitions in general. But it also highlighted how server appliances remained much more of a niche than envisioned by their more vocal proponents.

The knocks on server appliances then and now haven't changed much. EMC Global Marketing CTO Chuck Hollis lays out some of the negatives in a recent post.

It's not the first big appliance that causes the problem, it's when you have a fleet of them that you realize you've traded one class of headache for another.

None of them are built the same way. None of them manage the same way. None of them are supported the same way. None of them know how to work together in a cooperative fashion, and so on.

Want standardization at the different layers of an architectural stack? Sorry.

Want a pool of resources that can flow and flex to support whatever workload is at hand? Sorry, can't do.

Want to use the latest-greatest infrastructure technology from (choose your favorite vendor here)? Sorry about that as well.

Hollis sums up his case as follows: "You can see the nature of the trade-off, can't you? It's basically trading immediate gratification for a specific project versus creating long-term value through IT infrastructure."

EMC's interest in this debate is twofold.

The first is to push the notion of virtual appliances, pre-built virtual machine images that can be deployed on a virtual infrastructure. The idea is that an IT department can buy or build an encapsulated stack of software for a particular function and then deploy it across a server infrastructure of their own choosing. Given that EMC owns about 90 percent of VMware, its interest in promoting additional reasons to deploy server virtualization is obvious.

Virtual appliances also have a close affinity to cloud-computing infrastructure as a service. Amazon Machine Images (AMI) are a form of virtual appliance. And EMC is moving in this direction as well with Atmos.

The second reason that we see Chuck Hollis pushing back on the appliance concept is that we're seeing other large and powerful vendors, his competitors, promoting it. As opposed to the appliances of the Internet boom that mostly focused on network functions, this round is also, or even primarily, about heavy-duty business applications.

Oracle's Exadata is perhaps the canonical example of today's "Big Appliance," as Hollis phrases it. However, IBM has its own take on deploying and operating complex workloads such as business analytics. These may not be cookie-cutter appliances like a firewall or a Web server appliance. The tasks in question are too complex for that.

But they still bring together hardware and software from a single vendor and bundle them together. The marketing literature legitimately couches this integration in terms of customer benefits such as optimization. But such bundles also, and certainly not incidentally, increase a vendor's footprint and reduce the opportunity for others to capture a slice of the pie.

NEW YORK--It was a larger and cheerier crowd that attended this year's Red Hat's analyst day at the New York Stock Exchange on Tuesday.

That shouldn't be surprising. At last year's meeting on October 7, Red Hat management had the dubious honor of ringing the closing bell on a day that saw the Dow Jones Industrial Average drop over 500 points.

This meeting took place in a time of what's probably best described as cautious optimism about the state of the economy. And in the context of Red Hat financial results that have continued to show growth at a time when so many companies in IT industry and elsewhere have not.

For the quarter ending August 31, its profit jumped 37 percent relative to the year-ago quarter, besting analyst estimates.

The day included a fair bit of discussion related to financial minutiae, as you'd expect for an event pitched primarily for financial analysts. However, it also included an overview of Red Hat's strategy and its technical direction. Here are a few things that caught my eye.

Jim Whitehurst, Red Hat's CEO, spent a fair bit of time talking about what boils down to fine-tuning of its go-to-market execution such as:

• Value of subscription. This includes what he called "education and compliance," essentially a euphemism for getting people using Red Hat Enterprise Linux (RHEL) without paying for it to purchase a subscription. It also encompasses improving renewal rates for those cases where RHEL has been preloaded by a system builder and bringing a greater focus to articulating the value of RHEL relative to free substitutes such as CentOS.
• Routes to market. This refers to a continued build-out of the channel so that system integrators and others who recommend and install systems for less sophisticated customers will specify Red Hat as part of their solution. This stands in contrast to how, historically, Red Hat was mostly pulled into accounts by technically-savvy users and IT departments.

The message I took away from this is that Whitehurst isn't looking to change Red Hat's direction in any major way but sees a fair number of areas where more focused execution could lead to financial improvements. Later in the day, we also heard that Red Hat is taking a more systematic approach to which products it allocates development dollars for work such as internationalization.

For his part, Paul Cormier, executive vice president of products and technologies, reiterated Red Hat's belief that virtualization (which should be taken as hypervisor in this context) belongs in the operating system. This argument has been in evidence for a while as my fellow analyst Stephen O'Grady discussed after last year's event. 

It stands in stark contrast to VMware's desire to make the operating system irrelevant. Or, to put it another way, VMware's ambition to make the VMware ESX and ESXi hypervisors the model for a new type of operating system. This is too fraught a debate to tackle here; I largely agree with Stephen's take in his post.

However, one of the interesting outcomes of this battle is that Red Hat has been cozying up to Microsoft, the other big gun on the "hypervisors belong in the OS" side. This includes Red Hat's announcement Wednesday "that customers can now deploy fully supported virtualization environments that combine Microsoft Windows Server and Red Hat Enterprise Linux."

This sort of interoperability is certainly a customer desire and both Red Hat and Microsoft can legitimately present it in those terms without anyone smirking. However, the enemy of my enemy is also my friend, at least up to a point.

I also took note that Red Hat finally seems to be making some progress on the management front.

The product in question is RHEV Manager (RHEV-M); it's covered in detail in this video from the Red Hat Summit in September and is currently being tested by customers.

One reason I think it's important is that Red Hat apparently, if belatedly, recognizes that it is. CTO Brian Stevens admitted that RHEV-M "has been a huge missing ingredient."

The one customer speaker at the analyst day was Dave Costakos of Qualcomm and he focused on his company's experiences with testing RHEL-based virtualization and the associated RHEV Manager which he describes as "hits the mark."

I caught up with Dave at a break to get a bit more detail. He told me that they wanted a Web-based interface, which RHEV Manager has. He also liked the integration with Active Directory and other directory systems, and the role-based access controls. He said that it could perform the provisioning operations that Qualcomm requires and otherwise meets their needs.

Management has historically been a relatively weak part of Red Hat's offering that was mostly focused on updating packages. This is really a reflection of the broader Linux and open-source ecosystem in general. Projects like Nagios and, more recently, GroundWork notwithstanding, management doesn't play well to the strengths of open source. It touches too many parts of an IT infrastructure and requires too much cooperative work with the vendors making the things that need to be managed.

It's reasonable to ask whether Red Hat is too late to win big with RHEV Manager and its associated KVM-based virtualization play. But it had to attack management from some angle unless it was prepared to just cede that area of differentiation and potential point of control to system makers and others.

Finally, no technology discussion today would be complete without at least a mention of cloud computing. Brian Stevens jokingly called it a "shiny thing that people are looking at how to monetize."

The cloud discussion covered several angles, not least of which was standardization efforts such as Deltacloud. Like most other standardization efforts, this focuses on what is often called Infrastructure-as-a-Service; Amazon EC2 and S3 are perhaps the best known examples. Stevens admitted that it's going to be much harder to define a standard set of higher-level services (platform as a service in the vein of Microsoft Azure) that are portable.

Red Hat's distinctive play in the infrastructure cloud essentially circles back to its approach to virtualization. In cloud infrastructure as imagined by Red Hat, the operating system matters in important ways.

That's because applications matter; indeed, applications are ultimately what matter most. And in on-premise computing, one of Red Hat's greatest values and differentiators is the vast number of certified applications that it runs. This certification matters to users because, if they encounter a problem, it means that they can call the application vendor to get support. Otherwise, they'd get a "sorry, that's not a supported configuration."

One can argue whether the software layering of which the historical operating system is a part is the most appropriate choice for cloud computing. Fellow CNET blogger James Urquhart dives deeply into this topic in a pair of recent posts.

However, whether it's the way it should be or not, it is for now. And for Red Hat to be able to enable users to carry the certification of applications into a cloud model is a significant differentiation.

SAN FRANCISCO--General manager of Intel Architecture Group Sean Maloney's announcement of a reference design for a "micro server" during his Tuesday afternoon keynote at the Intel Developer Forum brought me a sense of deja vu.

Intel's Sean Maloney holding microserver.

(Credit: Intel)

He disclosed "a new ultra-low-voltage Intel Xeon 3000 series processor featuring a TDP (Thermal Design Power) of only 30 watts. To complement the broad range of dense and power-optimized platform offerings, Intel also demonstrated publicly for the first time a single-socket 'micro server' reference system which will help enable micro server innovation and future specification." Intel plans to ship the 30-watt dual-core chip in Q1 on 2010; a 45-watt quad-core version is set to ship immediately.

A reference system is primarily intended to demonstrate a concept. It provides a hands-on experience for partners and customers and therefore an opportunity to experiment with and fine-tune the basic approach. The microserver reference design will accommodate 16 server modules in a 5U-high (8.75-inch) chassis. The server boards are approximately 8-inches by 4.5-inches.

Jason Waxman, the general manager of high density computing at Intel, told me that they see the primary target for this class of system as "hosting companies that do a lot of white boxes." White boxes are systems that are often assembled in-house from component parts such as motherboards and cases. Waxman added that such companies nonetheless want many of the features associated with servers--such as memory with error correcting code (ECC).

In Intel's view of the world, microservers very much target service providers and companies that host busy Web sites and otherwise are associated with high-scale network computing. It sees this market as distinct from large high-performance computing (HPC) installations. Vendors such as HP tend to treat network computing and HPC as more of an overlapping customer group.

My deja vu when it comes to microservers relates to the fact that we've seen them before. They used to be called blades.

That's not to say that blade servers don't already exist today, but they've largely evolved into a much different concept from how they were initially conceived. The blades sold today by the likes of Cisco, Dell, HP, and IBM are about virtualization and integration. They pull together computing, networking, and storage and tightly integrate them both physically and through software. They are, in a sense, a form of scale-out consolidation.

Sun has largely eschewed this integration with their blade product line. However, Sun blades are heavily focused on high-performance computing--even to the point of integrating the HPC-centric InfiniBand interconnect on some of its products.

Rather, microservers hark back to the days of RLX Technologies, the company that did the most to promote blade servers during the Internet boom of circa 2000. Microservers are simply thin servers--compact, cheap, and simple. They provide cable simplification. They let hosting providers allocate low-cost physical servers to customers who don't want to share using virtualization.

Microservers bring blades back to their roots. Everything old is new again.