EMC hasn't exactly kept its fully automated storage tiering (FAST) a secret. The company has talked about the technology at analyst events and its global marketing CTO, Chuck Hollis, has blogged on the topic.
But now version 1 has officially launched, despite earlier reports that it wouldn't arrive until 2010. I'll get to why there have probably been some mixed signals about availability in a bit, but first let's look at what FAST is.
Different types of storage associated with computers perform relatively better or worse. Faster is usually better of course. But faster also tends to mean more expensive per unit of capacity. This relationship holds pretty generally. After all, if a technology were simultaneously slower and more expensive, no one would probably use it. There are some other relevant characteristics, such as permanence, removability, and so forth but price and performance are two of the big ones.
FAST automates the placement of data based on the way it is accessed. For example, a database index that is frequently read and written to will migrate to high-performance storage while older data that hasn't been touched for a while will move to slower, cheaper storage. The fundamental idea is that a relatively small amount of fast/expensive storage can let an application run almost as quickly as if all the storage were fast and expensive.
The concept is similar in some respects to Sun's storage pools in its ZFS file system, a component of Solaris.
Unsurprisingly, given that EMC tends to view storage as being in the middle of things in the data center, in its case, FAST lives in the array. Three of its product lines are supported: Symmetrix V-Max (high-end storage area network arrays), CLARiiON CX4 (mid-range storage area network arrays), and Celerra NS (file-based network-attached storage). The basic FAST concept is the same across these products, but details differ in how they are managed and in some low-level specifics.
This is because Symmetrix and CLARiiON come from largely separate technology roots--and because Celerra operates at the file rather the block level. However, EMC told me in a recent briefing that its goal in FAST version 2, slated for mid-2010, is to largely mask platform differences from users using management and other administrative interfaces.
And this is where I think some of the mixed signals about release dates come from. FAST v1 certainly brings interesting and useful capabilities to the market. However, in v1, Symmetrix and CLARiiON also only apply migration policies at the logical unit number (LUN) level, a concept analogous to a drive letter on a Windows PC likely to correspond to many gigabytes of storage. FAST v2 will enable the relocation of blocks of under 1 megabyte.
And v2 will also see additional important capabilities such as the introduction of chargeback accounting in Ionix ControlCenter for those organizations that want to more precisely allocate costs to different business units.
In short, without suggesting that v1 isn't fully baked, clearly v2 following in just six months or so will be a significantly more complete and integrated technology suite.
EMC is making a huge deal of FAST, as well they should. If you look at where different storage technologies sit today, change is a-brewing. Let me explain.
The idea of storage tiers aren't new. They historically featured tape as a major piece, but solid state (flash) drives have been around for a long time as well. Disks and disk arrays have also long used memory caches, sometimes backed up with batteries, to improve performance.
But the caches, on the one hand, were limited. In the case of disk arrays, they mostly served the purpose of minimizing the performance degradation associated with certain RAID (redundant array of inexpensive disks) configurations which store parity information that allows recovery in the event of a disk failure.
And the other parts of the hierarchy were rather manual. This was sometimes OK in the case of tape used to archive data according to some preset policy. But solid state long remained a niche. You just needed too much of it to gain its performance benefits. In addition, for a long time, bottlenecks in storage controllers and in the connection between server and storage limited the performance benefit of solid state anyway.
But some dynamics are changing today.
The first is that we've pretty much reached the performance limits of "spinning rust" (as storage folks like to jokingly call disks). Drives continue to get bigger certainly. But 15,000 rpm Fibre Channel disks aren't going to get a whole lot faster. Sure, we can always add more of them--and that helps some--but then you have wasted capacity, more power and heat, and higher costs.
Another is that tape is going away for many purposes. Yes, it will be a long slow decline, but that's the trend.
And solid state is getting cheaper. It remains significantly more expensive than disk drives for a given size but its now affordable in quantities that are interesting for mainstream commercial computing.
Add those together and techniques allowing enterprise fibre channel drives (and tape) to be largely replaced over time by a combination of solid state and capacity/power-use-optimized SATA or SAS disk drives start to look very interesting.
EMC's description of this new hierarchy is FAST, Thin, Small, Green, Gone. In other words, solid state for performance, a reduced number of active high performance disk drives, de-duplicated data, low-activity drives that are spundown when not in use, and final data that is purged when no longer needed.
This is certainly a long-term vision. Change does not happen quickly in enterprise storage. But it's starting to happen.
On the one hand, vendor analyst events are a good opportunity to spend focused time diving deep into individual products, roadmaps, and corporate initiatives. On the other, they're a useful forum for getting the feel of a company's overall zeitgeist in a way that narrower discussions don't. EMC's event, held last week in Franklin, Mass., was no exception.
(Credit:
EMC)
Perhaps the single thing that struck me most about the event as a whole was the full integration of VMware into the discussion as a whole. I've been following both companies since before EMC acquired VMware in 2003. In the years since, although there were the obligatory nods to joint development work and "better together," VMware aggressively maintained a distance that was hardly limited to the 3,000 miles between VMware's Palo Alto, Calif., headquarters and EMC in Massachusetts. VMware's presence at EMC analyst events was largely relegated to a few off-hand mentions and perhaps a desultory breakout session given by a junior marketing person.
This year couldn't have been more different. VMware was very much woven into just about every discussion and one of VMware's senior technologists shared a panel with representatives from EMC and Cisco Systems. One thing that has changed, of course, is the ouster of VMware founder and CEO Diane Greene in 2008. It was Greene who most vocally kept EMC at arm's length. It's also the case that virtualization is increasingly at the center of everything that EMC does, so how could VMware not be an integral part?
This pervasive virtualization theme carried through to EMC VP Jon Peirce's discussion about EMC's internal IT infrastructure as well. EMC IT is using VMware to virtualize as much as possible. This includes doing database testing on a Cisco Unified Computing System (UCS) in advance of a planned migration off Sun E25000 UltraSPARC-based servers.
An initial Virtual Desktop Infrastructure (VDI) deployment also uses UCS in the form of a vBlock--a preconfigured package that combines products from Cisco, EMC, and VMware. EMC has about 200 users on VDI today and expects to roll out to several thousand next year starting in their Franklin facility. VDI and associated forms of desktop virtualization are a favorite technology of CEO Joe Tucci, who would like to move toward a platform-agnostic client strategy.
The ultimate goal is what sometimes goes by BYOPC (Bring Your Own PC), in which employees provide their own notebook computers, perhaps purchased with the help of a stipend. Even today, many of the EMC execs at the event were sporting Macs, even though IT doesn't officially support them.
Another hot topic at the event was multi-tier storage, in this case automatic storage tiering that intelligently moves data between Flash-based storage and conventional disk drives. EMC's technology here is called FAST and will roll out on Symmetrix V-Max arrays.
Flash drives can be much faster than SATA disks--or even high-performance Fibre Channel drives--but they're also much more expensive on a per-GB basis. The idea behind FAST is to automate the placement of data based on the way its accessed. For example, a database index that is frequently read and written to will migrate to high performance flash while older data that hasn't been touched for a while will move to slower, cheaper disks.
Disks being used to store rarely accessed archival data can even be deduped, compressed, and even spun down to reduce overall data center power consumption. Tape isn't part of this vision; Tucci opined that "Backup to and recovery from tape is dead."
The idea of storage tiering isn't new. Hierarchical storage management (HSM) has been around for well over a decade. However, in practice, it's mostly ended up being about moving old files to tape for archive purposes. (EMC itself has a product in this vein: Legato DiskXtended.) FAST is something more transparent and more dynamic.
There are analogs between FAST and the storage pooling that is part of Sun Microsystems' ZFS filesystem. EMC argues that the function belongs on the storage device rather than the server because the array is where data access from multiple systems and applications come together.
It's unsurprising that EMC wants storage to be at the center of things. This is a company, after all, whose tagline is "where information lives." It is, however, worth remembering that this is a different lens through which to view the world than system vendors tend to choose--and, for that matter, than VMware chose historically.
Downtime is bad enough. But it's a really bad day when you lose data.
We've seen much gnashing of teeth over Microsoft's loss and slow recovery of data stored on T-Mobile Sidekicks. In its latest update, Microsoft says: "We continue to make steady progress, and we hope to be able to begin restoring personal contacts for affected users this week, with the remainder of the content (photographs, notes, to-do-lists, marketplace data, and high scores) shortly thereafter.
But data loss problems are hardly unique to Microsoft.
Social-bookmarking service Magnolia suffered a complete and largely unrecoverable corruption of its database back in March. Cloud storage provider SwissDisk is the latest to suffer a hardware failure related to data storage.
The company writes: "To prevent any future outages SwissDisk management has signed a contractual service agreement with a $40B company. For the first time this will enable SwissDisk to provide our customers with a 99.95 percent service level agreement. SwissDisk users will enjoy the peace of mind that comes with access to a truly world class, internationally deployed, multi-million dollar, extremely resilient storage infrastructure and internet network."
One would have thought that such risk mitigation procedures would have been better implemented before an outage occurred rather than after. A saying about barn doors and horses comes to mind. But be that as it may, what do these sorts of failures say about cloud-based storage in general? Certainly these sorts of problems are among the poster children that get trotted out when people want to take swipes at cloud storage.
I don't buy claims that these incidents make any sort of statement about cloud storage in the general case. They do however suggest some things to think about when we're considering storing data with third parties.
Reliability requirements both in terms of data access and, ultimately, how willing you are to tolerate loss even as a rare event. It's tempting to glibly state that "nothing's perfect" and at some level that truism is hard to dispute. However, as a practical matter, IT systems can be made very, very robust and reliable with many layers of fallback and protection in place. Think of the systems and networks used for stock transactions for example.
However, as Cisco's Christofer Hoff notes: "Utility, cost efficiencies, convenience trump security, privacy and even (degraded) availability. Cloud doesn't escape that logic." In other words, if you want those layers of fallback and protection, you have to pay for them. Security also often has usability tradeoffs but that's a discussion for another time.
Transparency. That said, one of the issues today is that it's often difficult for the buyers of cloud services to evaluate the back-end procedures that are in place to safeguard data and enhance availability. To be sure, it will always be partly a matter of trust that "best practices" are in place; gear by itself isn't enough. But the view into how data is being stored and protected even at a gross level is often limited at best.
Primary vs. secondary. When we're talking about data storage specifically, one important distinction is whether you're using cloud storage to preserve your only copy of important data or whether it's for backups--or even an offsite copy of an onsite backup. Reliability still matters in any case. If a backup is lost, there's a vulnerability window until it can be restored. And, of course, you need assurance that a valid data backup exists at all.
However, in general, service provider failures that involve only copies of data don't have the same impact as when they involve the only copy of your information. In the latter case, understanding a service provider's procedures becomes especially important. Indeed, in many cases, prudence suggests storing another copy onsite or with an alternate provider.
The specifics will depend on a great many factors. They will be familiar to anyone versed in backup and archive procedures. As with many things, cloud computing does represent changes to the way that companies and individuals are doing IT. But much remains the same.
Coming from a technical background, when I hear the word "standards," I tend to think of things that usually go by strings of numbers (802.11), inscrutable abbreviations (USB), or at best a slightly more melodious moniker dreamed up by a marketing department (WiMAX).
They may be codified by a standards body as part of a formal de jure process. They may evolve through the efforts of one or several companies and gain wide enough usage to be considered a de facto standard. Or, as with PDF, they may start life as a de facto standard that is later ratified by a standards body.
Whatever the particulars, such technical standards allow equipment or software from a variety of different sources to work together in predictable ways.
When people talk about cloud computing, these are the sorts of standards that they're usually talking about. The main issue being addressed is portability--the canonical example being moving an application developed for Amazon Web Services to some other vendor's service.
In the Amazon case, a subset of AWS has indeed become a sort of de facto standard. Eucalyptus is one example of implementing a subset of AWS outside of Amazon. Another approach is epitomized by RightScale, which has emerged as a dominant management platform for AWS; it has efforts under way to mask the differences between different cloud infrastructure providers at the management level.
Well-established Web standards of various types also play key roles in creating and accessing cloud-computing services.
So technical standards for cloud computing are indeed moving forward. However, I find that when I talk to consumers of cloud computing, they don't really care much at this point. I noted this previously after I interviewed Tobias Klauder at Razorfish.
That's not to say that people don't care about standards in cloud computing. I led a session at the Massachusetts Technology Council unconference on The Future of Software and the Internet last week to address the question of whether we need standards for cloud computing.
The participants argued that we do indeed need standards. But they had a different sort of standards in mind--something more akin to these definitions:
- an average or normal requirement, quality, quantity, level, grade, etc.
- those morals, ethics, habits, etc., established by authority, custom, or an individual as acceptable
They were looking for more transparency in understanding data protection schemes--backup procedures, recovery point objectives, and recovery time objectives. They wanted to understand what is private and what is not and how data is secured and exported. They wanted this to be communicated in a straightforward manner and, ideally, audited by a trusted third-party in some way.
The issues raised in my session were less about finely tuned service level agreements (SLAs) and more about simply understanding the characteristics of a service. One size does not fit all. Thus, an online backup service such as EMC's Mozy has to offer a variety of options--including whether a user supplies their own encryption key (which itself must be kept secure and protected) or let the service provide one. An organization with specific archive requirements would need other types of guarantees and capabilities.
These are all standards. Some may become true standards over time in the sense of codified best practices and metrics. However, it's clear to me that what users most want in the near-term is to better understand, in plain English, some of the basic practices followed by cloud-service providers.
I just ran across this video from Sun Microsystems that demonstrates a source of disk latency that we usually don't think about. It's short and quite amusing. Take a look.
But being the analyst that I am, I also wanted to highlight some of the pieces that make the demonstration possible.
The technology that's under the covers, measuring the disk latency and so forth, is DTrace, a component of Solaris, originally developed by Bryan Cantrill. By way of background, here's what I wrote upon its introduction in Solaris 10:
DTrace stands for "Dynamic Tracing" and, indeed, it's that dynamism that most distinguishes it from other approaches. A developer, administrator, or performance tuner uses the DTrace scripting language to dynamically establish monitoring points of interest, whether in the OS kernel or user processes.
A probe is a location or activity to which DTrace can bind a request to perform a set of actions, like recording a stack trace, a time stamp, or a function argument. Think of them as programmable software sensors that gather interesting information about the system, and report it.
DTrace in Solaris 10 comes with something like 37,000 predefined probes; users can also define their own. Probes come from a variety of kernel modules that Sun calls providers, each of which creates a unique type of instrumentation.
For example, there's a provider that creates a simple time-based counter (profile) and another to understand lock contention and other sorts of locking behavior (lockstat). But essentially any function entry or exit is a potential probe location; DTrace hot-patches the function entry point in memory to insert the probe.
DTrace is an incredibly powerful tool in that it can do all this monitoring and measuring while minimizing its impact on a running system. (SystemTap is a Linux analog that's considerably less mature.) This allows DTrace to be used in conjunction with production systems--and thereby look for performance bottlenecks under real loads rather than synthetic test cases. The downside of DTrace is that you have to be a bit of a Solaris performance guru to actually make effective use of it.
Enter Fishworks, which Bryan calls DTrace-based appliance analytics.
With analytics, we sought to harness the great power of DTrace: its ability to answer ad hoc questions that are phrased in terms of the system's abstractions instead of its implementation. We saw an acute need for this in network storage, where even market-leading products cannot answer the most basic of questions: "what am I serving and to whom?" The key, of course, was to capture the strength of DTrace visually--and the trick was to give up enough of the arbitrary latitude of DTrace to allow for strictly visual interactions without giving up so much as to unnecessarily limit the power of the facility.
You can also find a more detailed overview of the analytics in this Sun presentation (PDF).
Fishworks is a big part of the secret sauce and value-add of Sun's recently introduced Storage 7000 Unified Storage Systems. It also reflects Sun's new, or at least "tweaked," software strategy. While DTrace is open source (under the CDDL license) along with the rest of OpenSolaris, the GUI dashboard, and other analytics software that makes use of DTrace, is not. This is consistent with a more pragmatic approach to open source on the part of Sun that allows for keeping proprietary modules and components mostly aimed at large-scale production deployments.
Fishworks is one of those--and an impressive one at that. It will know if you shout at your disks!
- prev
- 1
- next
