Software, Interrupted

One of the cloud-related trends that developers have been paying attention to lately is the idea of "NoSQL," a set of operational-data technologies based on nonrelational technology.

These technologies do not replace the relational database but rather add a new tool to the developer toolbox. Business intelligence database technologies such as Aster Data, Greenplum, Neteeza, and Vertica do not completely replace the traditional relational database but rather use nonrelational databases to augment the software.

RedMonk analyst Stephen O'Grady wrote recently that NoSQL "adoption was inevitable because, just as in every other walk of life, there are different tools for different jobs in the technology world." NoSQL may not be exactly the right moniker, but the companies and developers behind these tools have legitimate substantiating points as to why the approach is right.

According to Dwight Merriman, CEO of 10gen (the commercial team behind the open-source MongoDB project), we'll see NoSQL complement existing applications for the foreseeable future.

The broad range of NoSQL tools that include projects like Cassandra, CouchDB, Hadoop, Memcached, and MongoDB bring to bear a number of technical advantages--even if no one tool does everything.

Horizontal scalability
Horizontal scalability, readily achievable for NoSQL solutions, fits incredibly well with cloud computing and general trends in computer architecture--toward more CPU cores rather than faster ones.

Performance
In some cases, the simplification of design of these solutions, as well as lack of normalization of the data, yields better performance. This often results in the developer not coding around the database.

Ease of assembly
Some NoSQL solutions facilitate easier software development. Mapping object data to JSON, a JavaScript data interchange format, is far less complex. The "schemaless" nature of many of these products is an excellent fit with agile development methodologies.

The typical software system of moderate complexity has many real and conceptual internal data stores. No one technology will be the right solution for all problems.

Forward-looking organizations should look at which technologies are appropriate for different data subsystems and begin to evaluate NoSQL technologies for appropriate projects.

One of the headlines that caught my eye today is this blog post from the Amazon Web Services team about a new Memcached as a service offering from Gear6.

For years, Memcached has been used quietly to provide in-memory caching for many popular Web sites, from LiveJournal (for which it was first developed six years ago) to Twitter, Flickr, and Facebook. In the last couple of months, Facebook has opened up about how it scales, and one the key technologies enabling this is Memcached, which services 120 million queries every second. To achieve this, Facebook admits it has had to do some engineering work to improve Memcached's performance and memory efficiency.

Similarly, Gear6 has added features such as replication, clustering, optimized memory utilization and management to create what it calls a Memcached distribution, much in the same manner as Linux distributions are packaged. Joaquin Ruiz, executive vice president of products at Gear6, provided me with additional insight into why Memcached is popular with Web 2.0 sites and why it matters for cloud computing.

The problem, according to Ruiz, is dynamic data services. In a recent blog post, he pointed to the tight connection between dynamic content and Web 2.0; that is, one defines the other. In this Web 2.0 world, the LAMP (and to some extent Java and Ruby) stack "provided a low-cost, efficient development foundation for Web 2.0 but did not free us from the monolithic, vertically oriented, "scale-up" platforms. Memcached provided the heavy lifting in terms of horizontally scaling ("scale-out") on non-monolithic SMP server architectures from Intel and AMD."

In the Facebook example, the Memcached tier stores members' personalized dynamic content, such as status updates, wall posts, etc., so that they can be quickly accessed when queried. It's a similar set up for Twitter tweets or comments on photos on Flickr. While latency in a social application is mildly annoying, latency in a transactional application could mean lost revenue.

Dynamic data services will likely remain an important part of cloud services, which brings us back to the idea of a Memcached service on a cloud platform. Amazon's Jeff Barr noted, "powerful, high-level services like this allow application developers to spend more time focusing on the novel and value-added aspects of their application and less time on the underlying infrastructure."

Anything that developers and companies can take advantage of to serve data faster and more efficiently means they have time to do other things, including increasing their bottom line.

information technology is expected to play an important part in the global economic recovery, according to a new survey released Wednesday.

Some 72 percent of business and information technology executives say their "organizations place greater value on the IT function today than they did before the economic crisis" and that they "view IT as an important part of their economic recovery efforts," according to Accenture's Global Survey on IT Investments.

This is not an unfamiliar sentiment and is one we've heard from United States CIO Vivek Kundra as he's attempted to use IT to kick start a variety of programs on the federal level that will set the pace for innovative new uses of technology across the globe.

The results of the Accenture survey are similar to last week's Goldman Sachs cautiously optimistic survey results that suggested IT spending would trend upward in 2010 and normalize to pre-recession levels with the majority of countries represented planning to increase investment selectively next year.

... Read more

Goldman Sachs' latest IT spending survey is out and it looks a tech-spending recovery is on the way for 2010. To a large extent, the data suggests not so much that spending is dramatically higher, but that it has normalized at pre-recessionary growth rates, rather than contracting as it has over the past several months.

Goldman is cautiously optimistic about 2010 spending, noting that much of it depends on the macro-economic environment driving more business spending. And while most areas will see growth counter to 2009's downward spiral, some areas such as off-shore development will feel significant retraction.

Regardless, the sentiments are positive and dramatically different than Goldman's report from November 2008 where IT spending was in a total death spiral. What a difference a year makes.

A few key points from the report:

• With recessionary buying cycle clearly through the trough, the remaining question centers on the pace of recovery for 2010.
• Infrastructure, application development, and systems integration remain top spending areas, especially as CIOs start to consider newer technologies such as virtualization and cloud computing.
• There is pent-up demand in hardware most notable, positive for storage and server/PC refresh.
• The appetite for offshore services appears to be below trend at current levels.
• HP, NetApp, CommVault, Red Hat, Riverbed, and Salesforce.com are notable names showing positive upward momentum in our latest survey.

In software, Red Hat and Salesforce.com showed strengthened results with VMware and Citrix remaining top of mind, which Goldman believes to be a good indication of internal and external cloud deployments gaining momentum.

... Read more

With infrastructure services like Amazon EC2, Rackspace, and VMware making it easy to take advantage of the flexibility, portability, and reduced costs of cloud computing, it seems obvious to jump on the cloud bandwagon for new IT projects.

But, developers are generally left on their own to deal with the pain of deploying their apps to the cloud: configuring application servers, libraries, disk partitions, networking, clustering, service connections, and virtual private networks. After they get their app installed they also need to install management agents that run on top of the application layer.

If you really want to take advantage of the cloud and optimize return on investment, you'll want the on-boarding process to be easy and fast and you won't install that agent. Agent-based solutions are inherently inflexible. Deploying agent-based solutions in a cloud-based environment, which is, by definition, highly flexible, is often like trying to fit a square peg in a round hole. In agent-based solutions, hard-coded agents are installed on every machine to monitor the application. If a change to the application configuration occurs--such as the IT department adds a node or upgrades a component--the agents must be updated as well.

Each agent and management server must be configured separately with management and monitoring solutions generally not portable. When every change to an environment requires installation of multiple agents on each server and configuration of multiple management servers, it becomes a tall order to move an application from a traditional infrastructure to the cloud, or from one cloud infrastructure to another: private to public, public to hybrid, or hybrid to private.

How do you get around this so you can actually capitalize on the benefits of cloud computing? Go virtual. Move application management, including easy on-boarding, from above the application stack into the underlying virtual layer, along with the rest of the cloud infrastructure.

I was recently briefed by webappVM CEO Isaac Roth on how the company is pioneering this new approach. He said the virtual path allows you to actually realize all of the flexibility, portability, and reduced costs that come with the promise of cloud computing.

... Read more

As consumers increasingly purchase sophisticated smartphones such as the iPhone, BlackBerry, and Droid, they are developing expectations for how these phones allow contacts, calendars, e-mail, and social networks to remain in sync across all their devices.

One of the big challenges is that users don't always maintain the same source of inputting data--they switch from browser to desktop application to smartphone as their data access and entry point, introducing many variables into the data chain. And data integrity will only get more complicated as more applications become browser-based and keep no local data storage.

Most enterprise users have a local store in addition to the cloud storage, something that I still find puzzling from the T-mobile Sidekick outage, where consumer data that should have been in multiple locations (or at least present on the device) was thought to be lost.

The most common sync services are not provided directly by the mobile operator. Generally this is a good thing, as the more you can dis-intermediate the carrier, the more control you have over your data. But because the sync services are provided by others--notably Microsoft, Google, and Apple--you end up locked-in to their data structures as well as whatever privacy and data management issues that might arise in relation to advertising or other usage of your information.

Today, you can fairly easily sync your mobile device with most common online e-mail and PIM services although the BlackBerry, Droid, and the iPhone differ in their approaches--or at least in the visibility of how they work. For example, you can sync with Gmail and other services on the iPhone, but it rather perversely requires the Microsoft ActiveSync protocol.

By controlling the address book, Google and Apple effectively lock-in users to their sync service, leaving the carriers and devices to be easily replaced (minus the cancellation charges.) The user would barely notice the difference, aside from the sticker on his phone that says AT&T or Verizon.

Mobile operators do not want to cede control of the address book to Google or Apple, but they are late to the game and do not yet have sync solutions of their own. As a result, they are scrambling to add this functionality, but building a sync solution that works with all different devices and email services is no easy task, thanks to the widespread problem of device fragmentation in the industry.

One option is to deploy a white label solution, like the open mobile cloud sync offered by Funambol. Funambol CEO Fabrizio Capobianco told me the company has been approached by many of the top mobile operators, with several of them looking to setup sync services for their customers. They all recognize the issue, and according to Capobianco can turn to Funambol as a way to quickly bring a high-quality solution to market.

With all the different players in mobile sync, users will begin to question who owns their data. Enterprise users, in particular, should have privacy concerns about trusting their data to someone else. In the case of Android users, there is a growing anti-Google sentiment, and if Google already owns your email, calendar, and search queries, do you really want them to own your phone contacts as well?

The cost benefits of virtualization are well-documented, allowing enterprises to significantly reduce the space and electrical power required to run data centers and streamline the management of an ever-growing number of servers.

Virtualization also provides means for expedient scalability. Given today's economic climate and cost-cutting mandates, it is not surprising that analyst firm Gartner recently predicted that 50 percent of workloads will run inside virtual machines by 2012.

What many organizations fail to understand, according to Amir Ben-Efraim, CEO of virtualization security provider Altor Networks, is that collapsing multiple servers into a single one with several virtual machines inside eliminates all firewall, intrusion detection, and other protections in existence. Physical security measures literally become "blind" to traffic between VMs, since they are no longer in the data path.

This echoes comments made by Gartner analyst Neil MacDonald, who wrote in a recent presentation titled "Securing the Next-Generation Virtual Data Center" (subscription required), that "most virtual machines you deploy will be less secure than the physical systems they replace," and that "virtualization will radically change how you secure and manage computing environments."

VMware recently launched a partner program to help ISVs develop solutions certified as "VMsafe." VMsafe provides API sharing through a secure container, enabling partner companies to access virtual environments. This virtual security technology provides fine-grained visibility over virtual-machine resources, including monitoring every aspect of the system with the ability to address previously undetectable viruses, rootkits, and malware before they can infect a system.

I spoke to Ben-Efraim to better understand the issues around VM security and for what users should be on the lookout. According to him, there are two common approaches that use existing methods to secure virtual-network traffic: using VLANs to separate and control communication between VMs; and taking software-based firewalls and running them as agents on each VM. Unfortunately, both of these approaches fall short.

VLAN segmentation extends the notion of LAN resource segmentation to include VMs. The approach essentially requires that VMs, which can naturally be grouped (i.e. by function or user base), be isolated from other VMs by use of virtual switches and routing (i.e. the human resources VLAN contains HR-serving VMs). However, VLAN segmentation is not a permanent solution to securing environments because of networking complexities, performance degradation, and security limitations of the approach, Ben-Efraim said.

... Read more

Microsoft released on Thursday a new position paper, "Privacy in the Cloud Computing Era: A Microsoft Perspective," that includes information about the remote storage and processing of personal information.

Privacy and security concerns continue to be a primary argument that cloud naysayers use against storing data and applications on the Internet. Big IT vendors and service providers like Microsoft and Hewlett-Packard will sooner or later be forced to take the cloud seriously or risk missing out on the whole next wave of IT consumption. And their large enterprise customers will expect them to offer cloud services with the appropriate levels of privacy and security measures in line with their business needs.

The interesting thing about this paper is that Microsoft takes surprisingly minimal responsibility for the data it will manage:

... Read more

IBM on Wednesday announced a program designed to help educators and students pursue cloud-computing initiatives and better take advantage of collaboration technology in their studies.

The IBM Cloud Academy, announced at the Educause annual conference, includes a global roster of educational institutions as initial participants. Educause is a nonprofit association whose mission is to advance higher education by promoting the intelligent use of information technology.

IBM will provide the cloud-based infrastructure for the program, with some basic collaboration tools available at the outset. IBM's LotusLive service provides the basis for the new offering. Participants will immediately be able to do some very basic tactical functions on the new system:

• Create working groups on areas of interest to the education industry
• "Jam" on new innovations for clouds in education-related areas with IBM developers
• Work jointly on technical projects across institutions
• Share research findings and exchange new research ideas

Shared research across universities and other higher-learning institutions remains a vital part of technological innovation, but many programs don't have formal tool sets in place. Cloud services are a logical place to run these types of programs, especially as international groups need immediate access to data from their partners.

... Read more