Retiring application data to the cloud

As the world pushes ahead with cloud computing and business users demand software-as-a-service (SaaS) applications, many IT departments are struggling to keep legacy applications on life support. Many of these zombie applications are there only for storage and audit purposes, not for real-time data interaction.

Even if applications have been "turned off" the data continues to live on in databases and file stores, continuing to take up storage space and software licenses. The result is a state of paralysis, with application retirement merely a dream.

U.K.-based Clearpace recently unveiled a cloud-based data archiving service called RainStor. RainStor's technology is being used to solve a completely overlooked problem domain: application retirement. I spoke via e-mail with RainStor CTO Andy Ben-Dyke to understand how the service works and why it makes sense.

RainStor's Instant Application Retirement service works in 3 steps:

1. Send--Structured data from any RDBMS is automatically compressed by 40x or more, encrypted and sent to the cloud using a client-side software appliance. The extreme compression that is applied significantly reduces the time to transfer large volumes of data to the cloud.

2. Store--The encrypted data is stored in a private archive on Amazon's highly available and secure storage cloud (S3). Though compressed, the original schema format is preserved and RainStor is able to layer on additional archives which reflect any schema changes (e.g. add or delete of columns).

3. Search--Running on Amazon's highly scalable compute cloud (EC2) RainStor allows you to query data through any industry-standard reporting or BI tools over ODBC or JDBC with lightning speed. Providing "point-in-time" query capability based on its ability to store schema evolution changes.

The RainStor service can be had for as little as $1 per GB of data stored per month with no commitments, including Amazon storage and resource costs. Clearpace is also offering a 90 day free trial.

Given that there is a untold fortune of hardware and software tied up in legacy apps waiting to freed up, turning off those apps and sending the data to "heaven" in the clouds just seems like a no-brainer.

Follow me on Twitter @daveofdoom

[Remo_Williams]

yep, it would seem like a no-brainer... for people with no brains. what about regulatory oversight of archived data to satisfy FDA/ NRC/ FAA auditors? "Oh, we store it in the cloud". "Really... okay, here's your 483. Good luck with the consent decree hearings..."

[mbenedict]

@Remo_Williams:<br /><br />I can't comment on the Clearpace / RainStore offering specifically, but your remarks seem completely off base.<br /><br />First, not all businesses are subject to stringent regulatory or industry compliance requirements.<br /><br />Second, even for business which are, there is NO underlying reason why moving solutions "to the cloud" could not satisfy compliance requirements.<br /><br />In fact, today many business subject to regulatory oversight have ALREADY moved applications &#38; data to the cloud. For example, some businesses have built HIPAA-compliant solutions using cloud services such as Amazon Web Services and Microsoft HealthVault.<br /><br />From an IS auditor's perspective -- which I am one by the way -- we would audit a system built-on or connected-to the cloud like any other system. We'd thoroughly look at all the system risks (including inherent risks of the cloud) and look at whether the system's internal controls effectively mitigate those risks. <br /><br />Many cloud providers are currently undergoing service organization audits (i.e., SAS70) to provide clients (and their auditors) insight into the provider's internal control environment. Some cloud providers are also taking steps to enable their clients' compliance with industry standards, most notably PCI DSS. Companies like Microsoft are prepared to enter into "BA agreements" with their clients, as required by HIPAA.

[tristan-slominski]

The problem with currently migrating the data to the cloud is that you have to trust your Cloud provider with the secrecy of your data, which gives you two choices. 1. trust the Cloud provider, and that they will not have disgruntled employees looking at what you search for in your data, or 2. encrypt everything and use the Cloud as a dead drop until you need to retrieve information. These two solutions do not cover every business need out there. What if you want to keep everything secret from the Cloud, yet still tap into the ability to search through live data without having to trust the Cloud?