EMC hasn't exactly kept its fully automated storage tiering (FAST) a secret. The company has talked about the technology at analyst events and its global marketing CTO, Chuck Hollis, has blogged on the topic.
But now version 1 has officially launched, despite earlier reports that it wouldn't arrive until 2010. I'll get to why there have probably been some mixed signals about availability in a bit, but first let's look at what FAST is.
Different types of storage associated with computers perform relatively better or worse. Faster is usually better of course. But faster also tends to mean more expensive per unit of capacity. This relationship holds pretty generally. After all, if a technology were simultaneously slower and more expensive, no one would probably use it. There are some other relevant characteristics, such as permanence, removability, and so forth but price and performance are two of the big ones.
FAST automates the placement of data based on the way it is accessed. For example, a database index that is frequently read and written to will migrate to high-performance storage while older data that hasn't been touched for a while will move to slower, cheaper storage. The fundamental idea is that a relatively small amount of fast/expensive storage can let an application run almost as quickly as if all the storage were fast and expensive.
The concept is similar in some respects to Sun's storage pools in its ZFS file system, a component of Solaris.
Unsurprisingly, given that EMC tends to view storage as being in the middle of things in the data center, in its case, FAST lives in the array. Three of its product lines are supported: Symmetrix V-Max (high-end storage area network arrays), CLARiiON CX4 (mid-range storage area network arrays), and Celerra NS (file-based network-attached storage). The basic FAST concept is the same across these products, but details differ in how they are managed and in some low-level specifics.
This is because Symmetrix and CLARiiON come from largely separate technology roots--and because Celerra operates at the file rather the block level. However, EMC told me in a recent briefing that its goal in FAST version 2, slated for mid-2010, is to largely mask platform differences from users using management and other administrative interfaces.
And this is where I think some of the mixed signals about release dates come from. FAST v1 certainly brings interesting and useful capabilities to the market. However, in v1, Symmetrix and CLARiiON also only apply migration policies at the logical unit number (LUN) level, a concept analogous to a drive letter on a Windows PC likely to correspond to many gigabytes of storage. FAST v2 will enable the relocation of blocks of under 1 megabyte.
And v2 will also see additional important capabilities such as the introduction of chargeback accounting in Ionix ControlCenter for those organizations that want to more precisely allocate costs to different business units.
In short, without suggesting that v1 isn't fully baked, clearly v2 following in just six months or so will be a significantly more complete and integrated technology suite.
EMC is making a huge deal of FAST, as well they should. If you look at where different storage technologies sit today, change is a-brewing. Let me explain.
The idea of storage tiers aren't new. They historically featured tape as a major piece, but solid state (flash) drives have been around for a long time as well. Disks and disk arrays have also long used memory caches, sometimes backed up with batteries, to improve performance.
But the caches, on the one hand, were limited. In the case of disk arrays, they mostly served the purpose of minimizing the performance degradation associated with certain RAID (redundant array of inexpensive disks) configurations which store parity information that allows recovery in the event of a disk failure.
And the other parts of the hierarchy were rather manual. This was sometimes OK in the case of tape used to archive data according to some preset policy. But solid state long remained a niche. You just needed too much of it to gain its performance benefits. In addition, for a long time, bottlenecks in storage controllers and in the connection between server and storage limited the performance benefit of solid state anyway.
But some dynamics are changing today.
The first is that we've pretty much reached the performance limits of "spinning rust" (as storage folks like to jokingly call disks). Drives continue to get bigger certainly. But 15,000 rpm Fibre Channel disks aren't going to get a whole lot faster. Sure, we can always add more of them--and that helps some--but then you have wasted capacity, more power and heat, and higher costs.
Another is that tape is going away for many purposes. Yes, it will be a long slow decline, but that's the trend.
And solid state is getting cheaper. It remains significantly more expensive than disk drives for a given size but its now affordable in quantities that are interesting for mainstream commercial computing.
Add those together and techniques allowing enterprise fibre channel drives (and tape) to be largely replaced over time by a combination of solid state and capacity/power-use-optimized SATA or SAS disk drives start to look very interesting.
EMC's description of this new hierarchy is FAST, Thin, Small, Green, Gone. In other words, solid state for performance, a reduced number of active high performance disk drives, de-duplicated data, low-activity drives that are spundown when not in use, and final data that is purged when no longer needed.
This is certainly a long-term vision. Change does not happen quickly in enterprise storage. But it's starting to happen.
On the one hand, vendor analyst events are a good opportunity to spend focused time diving deep into individual products, roadmaps, and corporate initiatives. On the other, they're a useful forum for getting the feel of a company's overall zeitgeist in a way that narrower discussions don't. EMC's event, held last week in Franklin, Mass., was no exception.
(Credit:
EMC)
Perhaps the single thing that struck me most about the event as a whole was the full integration of VMware into the discussion as a whole. I've been following both companies since before EMC acquired VMware in 2003. In the years since, although there were the obligatory nods to joint development work and "better together," VMware aggressively maintained a distance that was hardly limited to the 3,000 miles between VMware's Palo Alto, Calif., headquarters and EMC in Massachusetts. VMware's presence at EMC analyst events was largely relegated to a few off-hand mentions and perhaps a desultory breakout session given by a junior marketing person.
This year couldn't have been more different. VMware was very much woven into just about every discussion and one of VMware's senior technologists shared a panel with representatives from EMC and Cisco Systems. One thing that has changed, of course, is the ouster of VMware founder and CEO Diane Greene in 2008. It was Greene who most vocally kept EMC at arm's length. It's also the case that virtualization is increasingly at the center of everything that EMC does, so how could VMware not be an integral part?
This pervasive virtualization theme carried through to EMC VP Jon Peirce's discussion about EMC's internal IT infrastructure as well. EMC IT is using VMware to virtualize as much as possible. This includes doing database testing on a Cisco Unified Computing System (UCS) in advance of a planned migration off Sun E25000 UltraSPARC-based servers.
An initial Virtual Desktop Infrastructure (VDI) deployment also uses UCS in the form of a vBlock--a preconfigured package that combines products from Cisco, EMC, and VMware. EMC has about 200 users on VDI today and expects to roll out to several thousand next year starting in their Franklin facility. VDI and associated forms of desktop virtualization are a favorite technology of CEO Joe Tucci, who would like to move toward a platform-agnostic client strategy.
The ultimate goal is what sometimes goes by BYOPC (Bring Your Own PC), in which employees provide their own notebook computers, perhaps purchased with the help of a stipend. Even today, many of the EMC execs at the event were sporting Macs, even though IT doesn't officially support them.
Another hot topic at the event was multi-tier storage, in this case automatic storage tiering that intelligently moves data between Flash-based storage and conventional disk drives. EMC's technology here is called FAST and will roll out on Symmetrix V-Max arrays.
Flash drives can be much faster than SATA disks--or even high-performance Fibre Channel drives--but they're also much more expensive on a per-GB basis. The idea behind FAST is to automate the placement of data based on the way its accessed. For example, a database index that is frequently read and written to will migrate to high performance flash while older data that hasn't been touched for a while will move to slower, cheaper disks.
Disks being used to store rarely accessed archival data can even be deduped, compressed, and even spun down to reduce overall data center power consumption. Tape isn't part of this vision; Tucci opined that "Backup to and recovery from tape is dead."
The idea of storage tiering isn't new. Hierarchical storage management (HSM) has been around for well over a decade. However, in practice, it's mostly ended up being about moving old files to tape for archive purposes. (EMC itself has a product in this vein: Legato DiskXtended.) FAST is something more transparent and more dynamic.
There are analogs between FAST and the storage pooling that is part of Sun Microsystems' ZFS filesystem. EMC argues that the function belongs on the storage device rather than the server because the array is where data access from multiple systems and applications come together.
It's unsurprising that EMC wants storage to be at the center of things. This is a company, after all, whose tagline is "where information lives." It is, however, worth remembering that this is a different lens through which to view the world than system vendors tend to choose--and, for that matter, than VMware chose historically.
The debate over single-function server appliances versus general-purpose servers is a long-standing one.
Appliances first came onto the scene in the late 1990s during the first Internet boom. They focused on a particular task, such as Web serving, and were designed to be ready to install with minimum muss, fuss, or skill. This assembly line approach to server farms was to be the secret sauce that made possible infinite growth without infinite IT staff.
Cobalt Networks was perhaps the best known and most sophisticated of the companies to offer appliances. Sun Microsystems later acquired Cobalt and then failed to successfully integrate it. This arguably presaged the mixed history of subsequent Sun acquisitions in general. But it also highlighted how server appliances remained much more of a niche than envisioned by their more vocal proponents.
The knocks on server appliances then and now haven't changed much. EMC Global Marketing CTO Chuck Hollis lays out some of the negatives in a recent post.
It's not the first big appliance that causes the problem, it's when you have a fleet of them that you realize you've traded one class of headache for another.
None of them are built the same way. None of them manage the same way. None of them are supported the same way. None of them know how to work together in a cooperative fashion, and so on.
Want standardization at the different layers of an architectural stack? Sorry.
Want a pool of resources that can flow and flex to support whatever workload is at hand? Sorry, can't do.
Want to use the latest-greatest infrastructure technology from (choose your favorite vendor here)? Sorry about that as well.
Hollis sums up his case as follows: "You can see the nature of the trade-off, can't you? It's basically trading immediate gratification for a specific project versus creating long-term value through IT infrastructure."
EMC's interest in this debate is twofold.
The first is to push the notion of virtual appliances, pre-built virtual machine images that can be deployed on a virtual infrastructure. The idea is that an IT department can buy or build an encapsulated stack of software for a particular function and then deploy it across a server infrastructure of their own choosing. Given that EMC owns about 90 percent of VMware, its interest in promoting additional reasons to deploy server virtualization is obvious.
Virtual appliances also have a close affinity to cloud-computing infrastructure as a service. Amazon Machine Images (AMI) are a form of virtual appliance. And EMC is moving in this direction as well with Atmos.
The second reason that we see Chuck Hollis pushing back on the appliance concept is that we're seeing other large and powerful vendors, his competitors, promoting it. As opposed to the appliances of the Internet boom that mostly focused on network functions, this round is also, or even primarily, about heavy-duty business applications.
Oracle's Exadata is perhaps the canonical example of today's "Big Appliance," as Hollis phrases it. However, IBM has its own take on deploying and operating complex workloads such as business analytics. These may not be cookie-cutter appliances like a firewall or a Web server appliance. The tasks in question are too complex for that.
But they still bring together hardware and software from a single vendor and bundle them together. The marketing literature legitimately couches this integration in terms of customer benefits such as optimization. But such bundles also, and certainly not incidentally, increase a vendor's footprint and reduce the opportunity for others to capture a slice of the pie.
Coming from a technical background, when I hear the word "standards," I tend to think of things that usually go by strings of numbers (802.11), inscrutable abbreviations (USB), or at best a slightly more melodious moniker dreamed up by a marketing department (WiMAX).
They may be codified by a standards body as part of a formal de jure process. They may evolve through the efforts of one or several companies and gain wide enough usage to be considered a de facto standard. Or, as with PDF, they may start life as a de facto standard that is later ratified by a standards body.
Whatever the particulars, such technical standards allow equipment or software from a variety of different sources to work together in predictable ways.
When people talk about cloud computing, these are the sorts of standards that they're usually talking about. The main issue being addressed is portability--the canonical example being moving an application developed for Amazon Web Services to some other vendor's service.
In the Amazon case, a subset of AWS has indeed become a sort of de facto standard. Eucalyptus is one example of implementing a subset of AWS outside of Amazon. Another approach is epitomized by RightScale, which has emerged as a dominant management platform for AWS; it has efforts under way to mask the differences between different cloud infrastructure providers at the management level.
Well-established Web standards of various types also play key roles in creating and accessing cloud-computing services.
So technical standards for cloud computing are indeed moving forward. However, I find that when I talk to consumers of cloud computing, they don't really care much at this point. I noted this previously after I interviewed Tobias Klauder at Razorfish.
That's not to say that people don't care about standards in cloud computing. I led a session at the Massachusetts Technology Council unconference on The Future of Software and the Internet last week to address the question of whether we need standards for cloud computing.
The participants argued that we do indeed need standards. But they had a different sort of standards in mind--something more akin to these definitions:
- an average or normal requirement, quality, quantity, level, grade, etc.
- those morals, ethics, habits, etc., established by authority, custom, or an individual as acceptable
They were looking for more transparency in understanding data protection schemes--backup procedures, recovery point objectives, and recovery time objectives. They wanted to understand what is private and what is not and how data is secured and exported. They wanted this to be communicated in a straightforward manner and, ideally, audited by a trusted third-party in some way.
The issues raised in my session were less about finely tuned service level agreements (SLAs) and more about simply understanding the characteristics of a service. One size does not fit all. Thus, an online backup service such as EMC's Mozy has to offer a variety of options--including whether a user supplies their own encryption key (which itself must be kept secure and protected) or let the service provide one. An organization with specific archive requirements would need other types of guarantees and capabilities.
These are all standards. Some may become true standards over time in the sense of codified best practices and metrics. However, it's clear to me that what users most want in the near-term is to better understand, in plain English, some of the basic practices followed by cloud-service providers.
Sorely tempted as I was to do otherwise, I sat on my keyboarding fingers while the VMware saga unfolded yesterday--or at least I limited myself to posting some initial thoughts via Twitter. I know a lot of the personalities involved--I first met ousted VMware CEO Diane Greene in 2000--but I didn't feel I knew enough to discuss what happened in detail. Now, the day after, I don't claim to know exactly what happened within EMC's walls, but I've heard and surmised enough to feel comfortable offering some thoughts.
This wasn't predominantly about financial results. A lot of financial commentators have focused on a 2008 revenue forecast that is "modestly below" previous guidance and a stock price well below once high-flying levels. Come on. The company was still targeting about 50 percent year-over-year revenue growth. It's hardly Diane's fault that investors bid VMware stock up to unsustainable levels. Sure, at some level, financials played into this whole debacle, but only indirectly, insofar as stronger revenue forecasts or a higher market cap may have given Diane a stronger bargaining position.
Nor was this about strategy or execution failure on the part of VMware. One story puts it that: "A carefully considered opinion is that the EMC board doesn't believe Greene is the person who can take VMware to the so-called 'next level.'" Carefully considered perhaps. But I profoundly disagree. If anything, VMware has taken the lead in articulating the value of a software ecosystem that leverages a virtualized foundation but goes way beyond basic hypervisor functions. Perhaps VMware didn't badge it with some vaguely cool-sounding but largely content-free name; they just went with "Virtual Infrastructure." But that's not even a bad thing in a market where many IT shops equate VMware with virtualization. Microsoft has an airy-fairy feel good slogan ("Dynamic IT"), but in looking at all of the process, lifecycle, and other virtues dynamism could lead to, VMware is in many ways uniquely working them, while just about everyone else is mostly talking about them.
How about Microsoft? I've also heard suggestions that this somehow happened because Microsoft was starting to get its virtualization efforts on track. This is supposed to be a surprise? And what was Diane supposed to have done about this, rather than continuing to take virtualization to higher levels and into more functions as VMware has done? Drop a nuke on Redmond? VMware has actually played its hand against Microsoft quite well, leveraging a first-mover advantage rather than milking a lead.
So what's left? All the evidence suggests that Diane's ouster revolved around to what degree VMware would remain independent of EMC. As this Fortune article from last year indicates, this wasn't a new source of friction between VMware and its EMC parent. Writer Adam Lashinsky presciently notes:
The biggest headache just might be EMC. The two companies continue to have as little to do with each other as possible. Greene and her acolytes butt heads frequently with EMC's senior executives, who remain annoyed they cannot benefit more directly from owning VMware by selling its software.
VMware's position is unique among EMC acquisitions. EMC standard operating procedure is to aggressively integrate the companies it acquires, moving employees around and frequently putting EMCers in charge. Not so VMware. After some initial signs that some decision-making and various business processes were being pulled into EMC's Hopkinton headquarters, VMware went right back to operating quite independently out of Palo Alto. VMware even built its own snazzy new eco-friendly headquarters last year.
I have little doubt that this conflict was at least partly personality-driven from both sides. However, our own discussions with key VMware storage, system, and software partners back up Diane's long-held contention that an arms-length relationship between VMware and EMC after the acquisition was absolutely essential to maintaining goodwill and cooperation with those other partners.
Adding to this long-standing conflict was the question of whether VMware would completely spin itself out of EMC. (The earlier IPO was for only about ten percent of the company.) From Steve Lohr of The New York Times:
Ms. Greene was fired after she refused to resign or take another position at VMware, according to a VMware manager who asked not to be named because he was not authorized to speak publicly. The point of conflict, the person said, was that Ms. Greene had been pushing hard for VMware to be spun off early next year. After five years of ownership, a subsidiary can be sold off in an essentially tax-free transaction. EMC bought VMware for $635 million in cash in December 2003.
So, the bottom line seems to be that Diane finally pushed too hard for VMware independence with an EMC organization that, in no small part, has been longing to pull it in more closely. Our understanding is that EMC CEO Joe Tucci was one of the people who actually supported a degree of independence for VMware in the past. However, whatever the individual opinions of EMC execs and its largely-insider board, the deed got done.
And that's very unfortunate for EMC and VMware. I don't really buy Ashlee Vance's calling out of Joe Tucci as the personification of all things EMC in this characteristically snarky Register story, but I reluctantly concur with the rest of his analysis.
That's not to say EMC and new VMware CEO Paul Maritz will rush out and change the way VMware operates overnight. But there's too much evidence of EMC desires to more directly leverage VMware assets and revenue opportunities to believe that we'll continue to see the same sort of hands-off relationship that we saw under Diane Greene's leadership. If VMware becomes just another EMC product, first VMware's partners would be livid, then they would shift their attention toward other virtualization platforms, of which there are an increasing number of options--if none currently as mature and fully-featured as VMware's.
If that happens, I could only say: You blew it, EMC.
- prev
- 1
- next
