This is the time of year to take stock in where high-performance computing (HPC) sits and where it is headed. That's because the SC09 conference is taking place in Portland, Ore., this week and it's the biggest HPC conference around.
SC is an odd duck as conferences go. Last year it had more than 10,000 attendees and, yet, it's a largely volunteer-organized event in a world where trade shows of this scope are packaged by conference specialists or some specific corporation. Think the much-renamed LinuxWorld (run by IDG) or VMworld (run by VMware).
"SC" comes from supercomputing. Today's large computer complexes are typically not supercomputers in the sense of a specialized architecture only suitable for a specific type of technical computing. Rather, as Ashlee Vance notes in The New York Times, "The supercomputing world was long dominated by systems that required specialized chips, memory systems and networking technology. But about 10 years ago, researchers realized they could link thousands of cheaper machines running on mainstream chips and achieve pretty solid performance."
Thus an HPC event is no longer about supercomputers per se (although the term is still used as a convenient moniker for a collection of resources managed as a single entity in a single location). Rather it's about the computing components, the interconnects, the storage, and the software that ties everything together and the applications that run on top.
The Top500 nicely illustrates the evolution of HPC over time. This list, released twice annually, ranks the largest publicly acknowledged supercomputers--as the term is used today--on the basis of a somewhat simplistic, but objective, benchmark. The Top500 entries are certainly not typical of mainstream HPC; they're the biggest of the big. But they nonetheless provide some quantitative insight into important trends.
The newest iteration of the list was released Friday. There were no striking departures from the trends of the last few years, but there was some continued evolution that's worth taking note of.
The continued rise of InfiniBand. InfiniBand is a system interconnect that offers a higher performance alterative to the ubiquitous Ethernet. Although its initial backers envisioned a broader role for the technology, it's settled nicely into HPC and, to a lesser degree, back-end commercial data center functions like database clusters where low latency and high bandwidth are also paramount. (The Sun/Oracle Exadata appliance uses InfiniBand for example.)
(Credit:
TOP500.org)
InfiniBand's initial growth in HPC wasn't so much about displacing Ethernet as it was displacing the fractured collection of high-performance interconnects that preceded it. Myricom's Myrinet and Quadrics' QsNet were the most common of these, but there were many. This year InfiniBand is deployed on 181 of the Top500, a 28 percent increase from a year ago.
That's a striking increase clearly. But what is perhaps more striking is that about half that increase came at the expense of Ethernet rather than mopping up a variety of older or proprietary connection technologies. This shift started between 2007 and 2008 but was even more pronounced this year.
It's certainly possible that the next 10GbE generation of Ethernet, which today is essentially absent from the list, could again push Ethernet's numbers higher. However, whatever the specific technology, the message that I take away is that large computer clusters are starting to favor more optimized interconnects even if they and the components they connect are largely off-the-shelf.
And we see an analogous trend with the proliferation of blade servers as well. Blades, a more modular and pluggable approach to system design, have proven popular in many enterprises and midmarket companies, in part, because they help bring together computing, storage, and networking technologies into a single integrated whole. That type of integration isn't of much interest in HPC. Rather, blades play to HPC by offering high densities and reducing cable count and complexity.
In fact, among x86 servers at any rate, dominance is not too strong a word to describe the presence blades in the Top500. Consider just one vendor, Hewlett-Packard. HP has 208 ProLiant systems on the list. A full 203 of these, almost 98 percent, are ProLiant c-Class blades.
Collectively, these trends suggest what might be thought of as a trend toward building optimization around standardization. In the main, especially as one moves down from the very top of the list, the Top500 is composed mostly of systems using mainstream technologies such as x86, Linux, and standard interconnects. Clusters are the dominant architecture.
But we're increasingly not seeing mere rackmount servers connected by Gigabit Ethernet. As the systems on the Top500 list grow in capability, we're seeing more focus on how they're packaged, powered, and connected.
On the one hand, vendor analyst events are a good opportunity to spend focused time diving deep into individual products, roadmaps, and corporate initiatives. On the other, they're a useful forum for getting the feel of a company's overall zeitgeist in a way that narrower discussions don't. EMC's event, held last week in Franklin, Mass., was no exception.
(Credit:
EMC)
Perhaps the single thing that struck me most about the event as a whole was the full integration of VMware into the discussion as a whole. I've been following both companies since before EMC acquired VMware in 2003. In the years since, although there were the obligatory nods to joint development work and "better together," VMware aggressively maintained a distance that was hardly limited to the 3,000 miles between VMware's Palo Alto, Calif., headquarters and EMC in Massachusetts. VMware's presence at EMC analyst events was largely relegated to a few off-hand mentions and perhaps a desultory breakout session given by a junior marketing person.
This year couldn't have been more different. VMware was very much woven into just about every discussion and one of VMware's senior technologists shared a panel with representatives from EMC and Cisco Systems. One thing that has changed, of course, is the ouster of VMware founder and CEO Diane Greene in 2008. It was Greene who most vocally kept EMC at arm's length. It's also the case that virtualization is increasingly at the center of everything that EMC does, so how could VMware not be an integral part?
This pervasive virtualization theme carried through to EMC VP Jon Peirce's discussion about EMC's internal IT infrastructure as well. EMC IT is using VMware to virtualize as much as possible. This includes doing database testing on a Cisco Unified Computing System (UCS) in advance of a planned migration off Sun E25000 UltraSPARC-based servers.
An initial Virtual Desktop Infrastructure (VDI) deployment also uses UCS in the form of a vBlock--a preconfigured package that combines products from Cisco, EMC, and VMware. EMC has about 200 users on VDI today and expects to roll out to several thousand next year starting in their Franklin facility. VDI and associated forms of desktop virtualization are a favorite technology of CEO Joe Tucci, who would like to move toward a platform-agnostic client strategy.
The ultimate goal is what sometimes goes by BYOPC (Bring Your Own PC), in which employees provide their own notebook computers, perhaps purchased with the help of a stipend. Even today, many of the EMC execs at the event were sporting Macs, even though IT doesn't officially support them.
Another hot topic at the event was multi-tier storage, in this case automatic storage tiering that intelligently moves data between Flash-based storage and conventional disk drives. EMC's technology here is called FAST and will roll out on Symmetrix V-Max arrays.
Flash drives can be much faster than SATA disks--or even high-performance Fibre Channel drives--but they're also much more expensive on a per-GB basis. The idea behind FAST is to automate the placement of data based on the way its accessed. For example, a database index that is frequently read and written to will migrate to high performance flash while older data that hasn't been touched for a while will move to slower, cheaper disks.
Disks being used to store rarely accessed archival data can even be deduped, compressed, and even spun down to reduce overall data center power consumption. Tape isn't part of this vision; Tucci opined that "Backup to and recovery from tape is dead."
The idea of storage tiering isn't new. Hierarchical storage management (HSM) has been around for well over a decade. However, in practice, it's mostly ended up being about moving old files to tape for archive purposes. (EMC itself has a product in this vein: Legato DiskXtended.) FAST is something more transparent and more dynamic.
There are analogs between FAST and the storage pooling that is part of Sun Microsystems' ZFS filesystem. EMC argues that the function belongs on the storage device rather than the server because the array is where data access from multiple systems and applications come together.
It's unsurprising that EMC wants storage to be at the center of things. This is a company, after all, whose tagline is "where information lives." It is, however, worth remembering that this is a different lens through which to view the world than system vendors tend to choose--and, for that matter, than VMware chose historically.
Correction at 7:15 a.m. PST November 4: At one time, Red Hat had planned to ship an embedded KVM hypervisor based on Fedora. But the Red Hat Enterprise Virtualization Hypervisor uses the RHEL 5.4 kernel and thereby picks up the same hardware verification portfolio.
With Tuesday's release of Red Hat Enterprise Virtualization Hypervisor and Red Hat Enterprise Virtualization Manager (RHEV-M) for servers, the company has completed the first phase of a server virtualization rollout that effectively now puts KVM front and center. Red Hat released KVM commercially for the first time in September as part of Red Hat Enterprise Linux 5.4.
KVM is a server virtualization technology that Red Hat acquired when it bought Qumranet in 2008. Red Hat favors KVM over the other primary open-source hypervisor, Xen, for both business and technical reasons. (Although, as of version 5.4, Xen remains the default hypervisor for RHEL.)
The business reason is that, while Red Hat has made contributions to Xen, competitors are far more associated with the project. Novell, the owners of the only other major enterprise Linux distribution, ran especially hard with Xen early on. And Citrix, not a direct competitor but certainly a major virtualization player, bought XenSource, the commercial entity formed by Xen's creators.
From a technical perspective, Red Hat's issue is that it's hard to keep Xen and the Linux kernel in sync. Xen's a standalone hypervisor layer but it has deeply invasive hooks into the Linux kernel and, therefore, keeping the two working together takes a lot of development and testing effort. It's a bit reminiscent of how new versions of the Veritas file system had to be carefully matched to new versions of Solaris or HP-UX.
By contrast, KVM is kernel-based. This means that it is actually part and parcel of the Linux kernel rather than a quasi-independent piece of software. In part for this reason, it's KVM that is now included in the mainline Linux kernel as of version 2.6.20.
As of version 5.4, an instance of RHEL can host guest virtual machines running RHEL 5 and other operating systems including Windows Server 2008. This announcement adds Red Hat Enterprise Virtualization Hypervisor, something that is often referred to as an "embedded hypervisor." It uses the same RHEL 5.4 kernel as Red Hat's full enterprise distribution.
Embedded hypervisors have taken off more slowly than many of us expected. But all the major virtualization players offer one so Red Hat needed to as well.
From my perspective, the Red Hat Virtualization Manager is more significant. On the one hand, management is important to--indeed central to--virtualization. On the other hand, it's an area where Red Hat has lagged. CTO Brian Stevens admitted as much to me when we spoke at the company's financial analyst day last month when he said that RHEV-M "has been a huge missing ingredient."
Red Hat historically mostly focused on updating packages. This is a reflection of the broader Linux and open-source ecosystem in general. Projects like Nagios and, more recently, GroundWork notwithstanding, management doesn't play well to the strengths of open source because it's such a "high surface area" application. But Red Hat had to attack management from some angle unless it was prepared to just cede that area of differentiation and potential point of control to system makers and others.
RHEV-M is Red Hat's first step toward remedying this deficiency. It seems a necessary move especially given that KVM is likely to be used, at least initially, as part of a Red Hat software stack and therefore Red Hat pretty much has to support the tools to manage KVM if it's to gain any market traction.
That said, this is very much a first step. The initial product only manages KVM. Furthermore, the management server has to be running Windows Server 2003 which you would rightly think a rather odd decision from a company that is one of the pioneers of open source. (Apparently, this was a decision by Qumranet and Red Hat has not yet developed a version that can run on Linux.)
Red Hat has clearly prioritized getting a usable if limited product into customers' hands. They trotted out one such at their financial analyst day. Dave Costakos of Qualcomm was happy with what he saw. He told me that they wanted a Web-based interface, which RHEV Manager has. He also liked the integration with Active Directory and other directory systems, the role-based access controls, and the provisioning capabilities.
Overall, Red Hat's virtualization play remains less filled in than do the plays of others. But it's now started in a systematic way.
Over the years, we've seen a variety of approaches intended to meld multiple small servers into a single larger system. 3Leaf Systems is the latest. On November 3, it introduced a Dynamic Data Center Server (DDC-Server) for AMD Opteron processors. The DDC-Server combines a custom DDC-ASIC chip with software to create a symmetric multiprocessing server with 32 6-core AMD "Istanbul" processors and 1 terabyte of memory.
The system, together with the InfiniBand switch required to interconnect the server components, 8TB of storage, and 3Leaf's software, is priced at $250,000. A smaller $99,000 version is also available. However, these systems should be thought of primarily as proof of concepts intended to create proof points with customers and to provide system makers with a tangible product. 3Leaf's go-to-market plan is to sign up system original equipment manufacturers and sell them ASIC (application-specific integrated circuit) chips and software--not to itself be a seller of systems.
The basic concept behind 3Leaf's design has quite a few antecedents.
In the 1990s, Data General and Sequent came up with large Unix server designs that connected "standard high volume" (SHV) x86 modules with cables using a protocol from Dolphin Technology called SCI. The component modules were never as standard or high volume as the SHV term implied but the approach still reduced development costs and increased the flexibility of the system relative to the more monolithic designs that characterized most large SMP servers of the day.
More recently, Virtual Iron developed a distributed hypervisor that could not only subdivide a single server in the vein of server virtualization products like VMware's ESX Server, but could also meld multiple smaller systems into large ones on the fly. (Virtual Iron later abandoned its proprietary hypervisor in favor of Xen and was later absorbed by Oracle.)
ScaleMP's vSMP Foundation is the current product for aggregating x86 servers that is probably most comparable to 3Leaf's. To date, it's been primarily focused on high-performance computing. The key distinction is that, unlike ScaleMP, 3Leaf uses a custom ASIC in addition to software. Both companies are primarily focused on InfiniBand as their interconnect although there is nothing architectural to prohibit the use of 10-Gigabit Ethernet over time. From a technical perspective, 3Leaf is essentially layering its own coherency protocol on top of InfiniBand. The current product uses the same socket as the AMD processor. However, 3Leaf also has a license for Intel's QuickPath Interconnect.
3Leaf says that, by developing an ASIC that gets into coherent memory transactions at the cache level, they are able to get better performance across a wider range of workloads than a purely software-based approach can.
Performance has been a stumbling block with this approach historically.
An SMP server, however constructed, is characterized by the fact that it is a shared memory architecture. This means that any processor can directly access any memory in the system. In general, this makes for a simpler programming model than distributed memory architectures, such as clusters, in which a lot of the work associated with making sure you're working with latest data is shifted from hardware to software.
How quickly a given processor can get to the memory that it needs plays a big part in a system's performance. In fact, for some workloads such as database transaction processing, memory access times can be the single factor that most affects how fast a system is. As a result, traditional large server designs incorporated expensive hardware such as crossbars to keep memory traffics flowing across the entire system quickly.
Today's small servers have equally speedy and high-bandwidth memory links--indeed their compact footprint can help to reduce latency even further. However, once you combine multiple nodes, the time it takes for a processor to access memory on another node can rise dramatically. The exact numbers depend on many factors, including what else is going on in the system at the time. But, as a rule of thumb, it takes at least twice as long to access memory on another node than if it were local--and could take several multiples of that. In other words, memory access is non-uniform; NUMA is the term often used.
Over time, operating systems have gotten much better at keeping processing and associated memory physically close to each other. Certain workloads are also less sensitive to NUMA designs than others. Many HPC, analytics, and business intelligence applications involve fewer of the sort of memory updates that tend to drag down performance in NUMA architectures than does typical enterprise online transaction processing.
It's also the case that, today, large SMP is as much about having a large and flexible pool of hardware resources for server virtualization as it is about having a single large SMP image. Thus, in many respects, large SMP is increasingly about management rather than monolithic application performance. Which is one of the reasons that we're seeing a general trend towards modularity in all SMP designs.
Thus, the SHV approach to SMP system design arguably sits closer to the mainstream than it has in the past.
As 3Leaf's Shahin Khan told me, the key factors with this approach are it "had better be low cost and work." Performance has to be acceptable over at least an interesting subset of workloads and there can't be a significant price premium over the constituent systems and hardware. And ultimately, for 3Leaf, success will result from convincing one or more major system OEMs that the time has arrived to add a system or systems based on this approach.
It used to be that system management products were for the care and feeding of individual servers. They could deal with many of them, sure, and might even have had tools aimed at automating repetitive operations. But, fundamentally, they mostly looked at systems in isolation.
Enterprise management tools, on the other hand, looked at the IT infrastructure big picture. Sophisticated and complex, tools like CA's Unicenter, HP's OpenView, and IBM's Tivoli were the aggregation point for alerts and reports about the health of an organization's IT. But they rarely actually did anything; they watched for problems but it was other software or system administrators that had to actually swing into action.
The bottoms-up orientation of system management tended to win out over time. Enterprise management was never displaced--exactly. But it did long seem as if many of the products in the enterprise management space sat far from where the interesting action was in the data center.
However, today, we're seeing a shift to system management that happens at the level of the data center as a whole or at least a virtualized pool of systems and applications. Virtualization is one of the drivers here. Another is "private clouds" or, if you prefer, a more dynamic and services-oriented view of IT resources.
As a result, system management products are starting to take on more and more of the roles that were traditionally associated with enterprise management. We're also seeing systems management meet enterprise management in the middle, so to speak. IBM's VMControl announcement on October 20 is a case in point.
As IBM puts it in their release: "VMControl allows combinations of physical and virtual IBM servers to be managed as a single entity. This approach--known as system pooling--expands the benefits of virtualization by helping corporate data centers simplify complex management functions and better share and prioritize use of critical resources such as processing power, memory and storage."
The new product, IBM Systems Director VMControl Enterprise Edition, is focused on virtualized environments. It supports IBM's PowerVM and z/VM as well as x86 virtualization technologies such as VMWare, Hyper-V and open x86 virtualization solutions. IBM plans to first offer it on IBM Power Systems running AIX in December, 2009 with other platforms coming next year.
VMControl Enterprise Edition works in concert with Tivoli; IBM also announced "a new version of Tivoli Provisioning Manager that provides enhanced automation of the manual tasks of provisioning and configuring servers, operating systems, middleware, software applications, storage and network devices." As I've discussed previously, Tivoli is very much a central part of how IBM views cloud computing and therefore how it thinks about the evolution of the enterprise data center.
PowerVM itself, as its full name implies, is part of IBM's Systems Director family. This is IBM's systems management portfolio; rough counterparts are HP Systems Insight Manager, Dell OpenManage, and Sun xVM Ops Center. Systems Director has been the recipient of considerable development and marketing attention in recent years that have greatly improved its integration across IBM's disparate product lines as well as its overall functionality.
Virtualization is no longer just about server consolidation. It does that, sure, and thereby reduces the number of physical servers that an organization needs to purchase. But, especially in enterprises, it's increasingly as much about resource pools and services (such as disaster recovery) enabled by virtualization as it is about consolidation. And that makes the need for management more rather than less.
The debate over single-function server appliances versus general-purpose servers is a long-standing one.
Appliances first came onto the scene in the late 1990s during the first Internet boom. They focused on a particular task, such as Web serving, and were designed to be ready to install with minimum muss, fuss, or skill. This assembly line approach to server farms was to be the secret sauce that made possible infinite growth without infinite IT staff.
Cobalt Networks was perhaps the best known and most sophisticated of the companies to offer appliances. Sun Microsystems later acquired Cobalt and then failed to successfully integrate it. This arguably presaged the mixed history of subsequent Sun acquisitions in general. But it also highlighted how server appliances remained much more of a niche than envisioned by their more vocal proponents.
The knocks on server appliances then and now haven't changed much. EMC Global Marketing CTO Chuck Hollis lays out some of the negatives in a recent post.
It's not the first big appliance that causes the problem, it's when you have a fleet of them that you realize you've traded one class of headache for another.
None of them are built the same way. None of them manage the same way. None of them are supported the same way. None of them know how to work together in a cooperative fashion, and so on.
Want standardization at the different layers of an architectural stack? Sorry.
Want a pool of resources that can flow and flex to support whatever workload is at hand? Sorry, can't do.
Want to use the latest-greatest infrastructure technology from (choose your favorite vendor here)? Sorry about that as well.
Hollis sums up his case as follows: "You can see the nature of the trade-off, can't you? It's basically trading immediate gratification for a specific project versus creating long-term value through IT infrastructure."
EMC's interest in this debate is twofold.
The first is to push the notion of virtual appliances, pre-built virtual machine images that can be deployed on a virtual infrastructure. The idea is that an IT department can buy or build an encapsulated stack of software for a particular function and then deploy it across a server infrastructure of their own choosing. Given that EMC owns about 90 percent of VMware, its interest in promoting additional reasons to deploy server virtualization is obvious.
Virtual appliances also have a close affinity to cloud-computing infrastructure as a service. Amazon Machine Images (AMI) are a form of virtual appliance. And EMC is moving in this direction as well with Atmos.
The second reason that we see Chuck Hollis pushing back on the appliance concept is that we're seeing other large and powerful vendors, his competitors, promoting it. As opposed to the appliances of the Internet boom that mostly focused on network functions, this round is also, or even primarily, about heavy-duty business applications.
Oracle's Exadata is perhaps the canonical example of today's "Big Appliance," as Hollis phrases it. However, IBM has its own take on deploying and operating complex workloads such as business analytics. These may not be cookie-cutter appliances like a firewall or a Web server appliance. The tasks in question are too complex for that.
But they still bring together hardware and software from a single vendor and bundle them together. The marketing literature legitimately couches this integration in terms of customer benefits such as optimization. But such bundles also, and certainly not incidentally, increase a vendor's footprint and reduce the opportunity for others to capture a slice of the pie.
NEW YORK--It was a larger and cheerier crowd that attended this year's Red Hat's analyst day at the New York Stock Exchange on Tuesday.
That shouldn't be surprising. At last year's meeting on October 7, Red Hat management had the dubious honor of ringing the closing bell on a day that saw the Dow Jones Industrial Average drop over 500 points.
This meeting took place in a time of what's probably best described as cautious optimism about the state of the economy. And in the context of Red Hat financial results that have continued to show growth at a time when so many companies in IT industry and elsewhere have not.
For the quarter ending August 31, its profit jumped 37 percent relative to the year-ago quarter, besting analyst estimates.
The day included a fair bit of discussion related to financial minutiae, as you'd expect for an event pitched primarily for financial analysts. However, it also included an overview of Red Hat's strategy and its technical direction. Here are a few things that caught my eye.
Jim Whitehurst, Red Hat's CEO, spent a fair bit of time talking about what boils down to fine-tuning of its go-to-market execution such as:
- Value of subscription. This includes what he called "education and compliance," essentially a euphemism for getting people using Red Hat Enterprise Linux (RHEL) without paying for it to purchase a subscription. It also encompasses improving renewal rates for those cases where RHEL has been preloaded by a system builder and bringing a greater focus to articulating the value of RHEL relative to free substitutes such as CentOS.
- Routes to market. This refers to a continued build-out of the channel so that system integrators and others who recommend and install systems for less sophisticated customers will specify Red Hat as part of their solution. This stands in contrast to how, historically, Red Hat was mostly pulled into accounts by technically-savvy users and IT departments.
The message I took away from this is that Whitehurst isn't looking to change Red Hat's direction in any major way but sees a fair number of areas where more focused execution could lead to financial improvements. Later in the day, we also heard that Red Hat is taking a more systematic approach to which products it allocates development dollars for work such as internationalization.
For his part, Paul Cormier, executive vice president of products and technologies, reiterated Red Hat's belief that virtualization (which should be taken as hypervisor in this context) belongs in the operating system. This argument has been in evidence for a while as my fellow analyst Stephen O'Grady discussed after last year's event.
It stands in stark contrast to VMware's desire to make the operating system irrelevant. Or, to put it another way, VMware's ambition to make the VMware ESX and ESXi hypervisors the model for a new type of operating system. This is too fraught a debate to tackle here; I largely agree with Stephen's take in his post.
However, one of the interesting outcomes of this battle is that Red Hat has been cozying up to Microsoft, the other big gun on the "hypervisors belong in the OS" side. This includes Red Hat's announcement Wednesday "that customers can now deploy fully supported virtualization environments that combine Microsoft Windows Server and Red Hat Enterprise Linux."
This sort of interoperability is certainly a customer desire and both Red Hat and Microsoft can legitimately present it in those terms without anyone smirking. However, the enemy of my enemy is also my friend, at least up to a point.
I also took note that Red Hat finally seems to be making some progress on the management front.
The product in question is RHEV Manager (RHEV-M); it's covered in detail in this video from the Red Hat Summit in September and is currently being tested by customers.
One reason I think it's important is that Red Hat apparently, if belatedly, recognizes that it is. CTO Brian Stevens admitted that RHEV-M "has been a huge missing ingredient."
The one customer speaker at the analyst day was Dave Costakos of Qualcomm and he focused on his company's experiences with testing RHEL-based virtualization and the associated RHEV Manager which he describes as "hits the mark."
I caught up with Dave at a break to get a bit more detail. He told me that they wanted a Web-based interface, which RHEV Manager has. He also liked the integration with Active Directory and other directory systems, and the role-based access controls. He said that it could perform the provisioning operations that Qualcomm requires and otherwise meets their needs.
Management has historically been a relatively weak part of Red Hat's offering that was mostly focused on updating packages. This is really a reflection of the broader Linux and open-source ecosystem in general. Projects like Nagios and, more recently, GroundWork notwithstanding, management doesn't play well to the strengths of open source. It touches too many parts of an IT infrastructure and requires too much cooperative work with the vendors making the things that need to be managed.
It's reasonable to ask whether Red Hat is too late to win big with RHEV Manager and its associated KVM-based virtualization play. But it had to attack management from some angle unless it was prepared to just cede that area of differentiation and potential point of control to system makers and others.
Finally, no technology discussion today would be complete without at least a mention of cloud computing. Brian Stevens jokingly called it a "shiny thing that people are looking at how to monetize."
The cloud discussion covered several angles, not least of which was standardization efforts such as Deltacloud. Like most other standardization efforts, this focuses on what is often called Infrastructure-as-a-Service; Amazon EC2 and S3 are perhaps the best known examples. Stevens admitted that it's going to be much harder to define a standard set of higher-level services (platform as a service in the vein of Microsoft Azure) that are portable.
Red Hat's distinctive play in the infrastructure cloud essentially circles back to its approach to virtualization. In cloud infrastructure as imagined by Red Hat, the operating system matters in important ways.
That's because applications matter; indeed, applications are ultimately what matter most. And in on-premise computing, one of Red Hat's greatest values and differentiators is the vast number of certified applications that it runs. This certification matters to users because, if they encounter a problem, it means that they can call the application vendor to get support. Otherwise, they'd get a "sorry, that's not a supported configuration."
One can argue whether the software layering of which the historical operating system is a part is the most appropriate choice for cloud computing. Fellow CNET blogger James Urquhart dives deeply into this topic in a pair of recent posts.
However, whether it's the way it should be or not, it is for now. And for Red Hat to be able to enable users to carry the certification of applications into a cloud model is a significant differentiation.
SAN FRANCISCO--General manager of Intel Architecture Group Sean Maloney's announcement of a reference design for a "micro server" during his Tuesday afternoon keynote at the Intel Developer Forum brought me a sense of deja vu.
Intel's Sean Maloney holding microserver.
(Credit: Intel)He disclosed "a new ultra-low-voltage Intel Xeon 3000 series processor featuring a TDP (Thermal Design Power) of only 30 watts. To complement the broad range of dense and power-optimized platform offerings, Intel also demonstrated publicly for the first time a single-socket 'micro server' reference system which will help enable micro server innovation and future specification." Intel plans to ship the 30-watt dual-core chip in Q1 on 2010; a 45-watt quad-core version is set to ship immediately.
A reference system is primarily intended to demonstrate a concept. It provides a hands-on experience for partners and customers and therefore an opportunity to experiment with and fine-tune the basic approach. The microserver reference design will accommodate 16 server modules in a 5U-high (8.75-inch) chassis. The server boards are approximately 8-inches by 4.5-inches.
Jason Waxman, the general manager of high density computing at Intel, told me that they see the primary target for this class of system as "hosting companies that do a lot of white boxes." White boxes are systems that are often assembled in-house from component parts such as motherboards and cases. Waxman added that such companies nonetheless want many of the features associated with servers--such as memory with error correcting code (ECC).
In Intel's view of the world, microservers very much target service providers and companies that host busy Web sites and otherwise are associated with high-scale network computing. It sees this market as distinct from large high-performance computing (HPC) installations. Vendors such as HP tend to treat network computing and HPC as more of an overlapping customer group.
My deja vu when it comes to microservers relates to the fact that we've seen them before. They used to be called blades.
That's not to say that blade servers don't already exist today, but they've largely evolved into a much different concept from how they were initially conceived. The blades sold today by the likes of Cisco, Dell, HP, and IBM are about virtualization and integration. They pull together computing, networking, and storage and tightly integrate them both physically and through software. They are, in a sense, a form of scale-out consolidation.
Sun has largely eschewed this integration with their blade product line. However, Sun blades are heavily focused on high-performance computing--even to the point of integrating the HPC-centric InfiniBand interconnect on some of its products.
Rather, microservers hark back to the days of RLX Technologies, the company that did the most to promote blade servers during the Internet boom of circa 2000. Microservers are simply thin servers--compact, cheap, and simple. They provide cable simplification. They let hosting providers allocate low-cost physical servers to customers who don't want to share using virtualization.
Microservers bring blades back to their roots. Everything old is new again.
VMworld, which took place last week in San Francisco, was hopping.
In fact, the number of attendees appeared to overwhelm many of the conference's educational labs in the early going. And the many vendors I spoke with at the event were happy about their booth traffic and the show in general. Now that I've had a bit of time to digest and distill three days of whirlwind activity, five points stick with me:
1. We're seeing virtualization--as technology, products, and solution sets--start to mature in many respects. Or at least the current phase of it. Fellow analyst Judith Hurwitz described how she "was left with the feeling that we are in between generations of technology at this year's VMworld."
For me, this conclusion comes out of the observation that there was relatively little on display related specifically to server virtualization that was fundamentally new and different. That's not to suggest a lack of vendor activity. Anything but. However, the activity largely took the form of new iterations and building out on existing templates.
2. Legitimate cloud computing was much in evidence. But, my, the cloud washing was fierce. Many, many companies offering management and other products relating to virtualized infrastructure were eager to present themselves as playing in the nebulously defined "private cloud" space.
VMware itself was as guilty as anyone. With the latest version of its virtual infrastructure product, now dubbed vSphere, already launched back in April, VMware's focus at the show tilted heavily toward cloud computing.
While there were a few specifics, such as vCloud Express, much of this took the form of forward-looking generalities. For example, VMware gave a lot of airtime to the notion of hybrid clouds that bridge private and public networks even though this is largely an architectural theory at this point.
3. So was anything both new and real? Yes. A couple things. One was I/O virtualization, which can be thought of conceptually as separating computing from I/O (network and storage connections mostly) and allowing that I/O to be shared and dynamically allocated. It's not really a new concept. Like many things, it has its roots in the mainframe and has, more recently, found a home in blade designs from the likes of Cisco Systems, Hewlett-Packard, and IBM.
However, the current crop of products are intended to work with standard rackmount or blade servers. Xsigo uses InfiniBand-based technology. Virtensys, Aprius, and NextIO essentially just bring PCI Express out of the server. This is a relatively young technology area but one worth watching.
4. Client-side virtualization was also a hot area even with Citrix and Microsoft--in many respects the top dogs in this space--in semi-exile from the show.
It's an exciting and evolving landscape with many new approaches and products. This includes work on protocols to improve the user experience over network connections of different types; Wyse, once exclusively a thin client purveyor, is now heavily focused here. We're also seeing a general trend toward making more effective use of the processor and graphics resources on the client rather than making the server and network do all the work; Wanova is a start-up that made an announcement shortly before the show in this area.
5. My last takeaway is a sort of meta point. The way that we do computing is changing in rather significant ways and virtualization--together with its related but distinct cousin, cloud computing--is at the focus of that change.
This is fundamentally a change in how we operate computer systems rather than, say, how we write software for them. However, because it ultimately affects how applications get delivered and computing is accessed, it has far broader implications than for just data center operators.
IBM's industry analyst meeting last week in Austin, Texas, covered the present and the future of its Power line. This is the system lineup once called the RS/6000 and pSeries into which was more recently folded the iSeries (previously AS/400, System 36, etc.) to form a new family called IBM Power Systems.
For our purposes here I am going to focus on Power in the guise of IBM's RISC-based lineup running a combination of AIX (IBM's flavor of commercial Unix) and Linux (either natively or using PowerVM Lx86 to run x86 Linux applications). IBM i, the successor to OS/400, also runs on the unified Power line.
IBM is slated to publicly unveil some of the details of Power7 , the next generation in the Power microprocessor lineup, at the Hot Chips conference at Stanford University on Tuesday. At the top-level, IBM describes Power7 as having eight cores per chip, four threads per core using simultaneous multithreading (SMT), and two to three times the per-chip performance of Power6 using the same amount of power.
Specifics of the systems based on Power7 will come later. IBM did say it'll build Power7 systems that generally add the required memory, bandwidth, networking, and so forth to keep the systems balanced.
Beyond future hardware, though, there were a few themes from this meeting worth highlighting.
IBM's Unix business is a smooth running machine at this point. We repeatedly saw a chart detailing IBM Unix Server revenue share gains at the expense of HP and Sun. The three companies held roughly equal share at the start of 2005; today IBM holds about 37 percent of the total market, about 10 percent more than those competitors.
One can always argue about which particular slicing and dicing of data most accurately represents the state of the market--of course, each vendor chooses the view that paints it in the best light--but it's hard to argue against the idea that IBM Power is on a nice upward trajectory.
Part of this is IBM's processor road map. One of the corollaries of Moore's Law is that time to market is performance. What would be a truly compelling processor in Year X is much less so in Year X+1 and even less so in Year X+2. And IBM Power development has been on a solid three-year cadence for major releases combined with interim speed bumps. Competitively, Sun's "Rock" has been reportedly canceled and Intel's next major iteration of Itanium, "Tukwila," has been much delayed.
However, it's also a case that IBM has simply made advancing Power Systems a real priority and has systematically put the resources in place to make that happen. I'm inclined to argue that this focus on scale-up, mission-critical, and integrated stacks that have helped lead to Power's success and the mainframe's renaissance have worked against IBM is the less differentiated segments of the x86 business. (IBM does bring many of the same concepts and even technologies to x86 with its higher-end eXA products.) Be that as it may, when it comes to Power Systems, IBM's understanding of the needs of large enterprise data centers for their core applications has paid off.
Some of this may seem unremarkable. It is IBM we're talking about here after all. However, the idea of a Unix server as an integrated stack or solution is actually a relatively new one in historical terms. Before x86 servers became so ubiquitous it was the Unix server that epitomized the roll-your-own style of computing. And Sun, until quite latterly, painted lack of any forced software (much less services) integration as a virtue.
Thus, while Power Systems are certainly still widely used in high-performance computing where the ethos still tends toward more home-grown customization and integration, Power Systems have generally evolved from a traditional Unix server mindset to one more familiar to mainframe buyers and operators. This isn't to say that Power Systems slavishly imitates all the approaches that its System z mainframe brethren take. After all, if a buyer wants a mainframe, IBM already makes one; no need to re-create a second version.
But concepts such as the benefits of owning many levels of the hardware and software stack are twined throughout Power Systems presentations--and this is very much a mainframe characteristic.
Perhaps this shines through most closely with virtualization, which is a central theme to Power Systems discussion.
- Power servers are virtualized. That doesn't mean that they have an embedded hypervisor that can be booted up or that they're factory-loaded with virtualization in some way. Rather, even if you choose to run just one copy of an operating system, you're running it virtualized. IBM's 6 million transactions per minute on the TPC-C benchmark was on a virtualized system. Virtualization is inherent in the processor and firmware in addition to the IBM management software that sits on top.
- Power Systems users make use of virtualization. As measured by the sales of PowerVM--IBM's suite to deploy, manage, and utilize multiple VMs on a server--about 66 percent of Power customers use or at least plan to use the features of virtualization at some level. (PowerVM comes in several editions; more money gets you more features.) Contrast this with x86 where, for all the legitimate interest in and excitement about virtualization, the penetration with new servers is still something in the 15 percent or so range.
- The largest Power servers are huge. With rare exceptions and bragging-rights benchmarks notwithstanding, big servers these days aren't about running one workload or even about running one operating instance. It's about running many, many workloads with the appropriate combination of workload management techniques to enforce separation, provide flexibility, and offer mechanisms to guard against failures of various kinds. Virtualization (in several forms) is the foundation for all of this.
It wasn't that long ago, even by the standards of the computer industry, the IBM's RISC/Unix lineup was in danger of becoming a sideline. IBM dabbled in Unix "unification" efforts and with Intel's Itanium processor, and invested heavily in work associated with scaling up Linux. These efforts often seemed aimed at marginalizing its own RISC/Unix processor and operating system development work over the long term. However, in 2001, IBM released Power4. It was a major step forward in processor performance, and had two CPUs per processor die--then an unusual feature. Around the same time, IBM also doubled down on the development of its AIX flavor of Unix.
The result of those decisions, combined with consistent execution, is a very good one for IBM.
