Beyond Binary

The massive data failure at Microsoft's Danger subsidiary threatens to put a dark cloud over the company's broader "software plus services" strategy.

A key tenet of that approach is that businesses and consumers can trust Microsoft to reliably store valuable data on their servers.

T-Mobile Sidekick Slide

(Credit: Corinne Schulze/CNET)

A week ago, though, Microsoft's Danger unit experienced a huge outage that left many T-Mobile Sidekick users without access to their calendar, address book, and other key data. That's because the Sidekick keeps nearly all its data in the cloud as opposed to keeping the primary copy on the devices themselves.

Things got even worse on Saturday, as Microsoft said in a statement that data not recovered thus far may be permanently lost. It's not immediately clear how many people lost their data. The outage earlier in the week affected a broad swath of Sidekick users, though many had data return during the week.

While outages in the cloud computing world are common (one need only look at recent issues with Twitter or Gmail), data losses are another story. And this one stands as one of the more stunning ones in recent memory.

The Danger outage comes just a month before Microsoft is expected to launch its operating system in the cloud--Windows Azure. That announcement is expected at November's Professional Developer Conference. One of the characteristics of Azure is that programs written for it can be run only via Microsoft's data centers and not on a company's own servers.

It should be pointed out that the Azure setup is entirely different from what Danger uses: the Sidekick uses an architecture Microsoft inherited rather than built (Microsoft bought Danger last year). Still, the failure would seem to be enough to give any CIO pause.

Update, 2 p.m. PT, 10/11/2009: I asked Microsoft for comment Saturday when I was writing this, in particular as to how the rest of its cloud might differ from the Danger set up.

Microsoft said Sunday that its the fabric controller that manages the Azure service is built with redundancy in mind.

"We write multiple replicas of user data to multiple devices so that the data is available in a situation where a single or multiple physical nodes may fail," Windows Azure general manager Doug Hauger said in a statement to CNET News.

That doesn't mean Azure is immune from data loss, though I'm told an entire data center would have to be wiped out, as opposed to just a server or collection of servers. I'd be interested to know whether Microsoft will also offer multiple location options so that users that want to can have their data in more than one physical spot as well.

But that's just one of many questions raised by this spectacular failure. Among the other questions still looming large in my head are:

1. What backup procedures did Danger have?

2. Just how many of T-mobile's Sidekick customers lost their data? (Feel free to let me know, Sidekick users.)

3. What impact will this have on the Pink project, which was largely seen as the evolution of the Sidekick, and some say was already in trouble?

4. Will this hurt Microsoft's efforts to build a brand around the notion of Windows Phone even though that uses a different architecture (with its own challenges, to be sure)?

Ray Ozzie is a big believer in the cloud. But he knows that large businesses don't yet share his confidence.

"Enterprises will not really trust the cloud until they get some experience with it," Ozzie said, during a speech at a J.P. Morgan investment conference in Boston on Wednesday. He said that large businesses are more likely to start by going with an online version of a familiar product like Microsoft Exchange than they are today to move a major piece of their business into the cloud. A Webcast of his speech is available on Microsoft's investor relations page.

In October, Microsoft announced Windows Azure, a set of tools that is somewhat akin to a Web-based operating system that developers can use to build software that can then run in Microsoft's data centers. The software is now in testing, with large businesses mostly just kicking the tires at this point.

"In the next year or two I believe that the biggest impact of cloud computing is going to be in things like Exchange and SharePoint for us or those comparable offerings from our competitors," Ozzie said. Using one of those services allows a company to know how much bandwidth they need to communicate with the cloud, understand how cloud services can be managed, as well as just build up a certain comfort level.

"It will work its way into other parts of the enterprise IT environment over time as they get their comfort level," he said.

Ozzie

(Credit: Microsoft)

One of the lighter moments came when Ozzie was asked what were the lessons Microsoft learned from Windows Vista.

"How much time do you have?" Ozzie quipped.

Ozzie then went on to discuss some of the problems with Vista, including the false starts that he said resulted from "overcommitment."

"We had a vision that was larger than what we could achieve within the period of time that we needed to bring (the product) to market," Ozzie said.

And by changing its timing and feature set, Ozzie noted that Microsoft's partners were both too early and too late when it came to deciding when to spend time on Vista.

"If we don't give very clear predictable signals to those partners...about dates," Ozzie said, "they don't know when to invest and when not to invest."

The result, he said, was that drivers weren't ready, leaving PC makers in a tough position and ultimately creating a less-than-satisfactory experience for consumers and businesses. Many of those issues, he said, were taken to heart when it came to planning and communicating around Windows 7, he said.

Some of Ozzie's more intriguing comments came when he talked about the need for partnership over time as Microsoft builds out its cloud. So far, Microsoft has built its own data centers, but they have largely been in the U.S. Because of varying regulations in different countries, though, Ozzie talked about the need for data centers "everywhere on earth."

"Every country will have data centers," he said, but added that Microsoft itself doesn't have the resources to build a cloud in each country. "We have to have partners."

Microsoft is again trying to convince the partners that sell its software that they can make money in a world in which customers are getting their software as a service directly from Microsoft.

At its annual partner conference, which is taking place this week in Houston, Microsoft offered more details on the finances that buttress that claim. For example, partners that sign up customers for the new $15-per-month bundle of hosted SharePoint, Exchange and Office Communications Server can get a 12 percent referral fee. The partners can also get a 6-percent cut of renewal fees provided they continue to be associated with the customer and get positive evaluations.

"We expect that to be a competitive entrant," Microsoft partner program VP Allison Watson said in a telephone interview following Tuesday's keynote speech

There's also room for companies to make money hosting Microsoft's products, she said. The key, she said, is for companies that want to host the same products Microsoft does to make sure they are either tailoring their product to a specific market, combining it with other elements of their customer's workflow or offering additional uptime guarantees that aren't part of Microsoft's product. "Our partners are not offering just a vanilla service," Watson said.

As part of Tuesday's keynote, partners also got to see a series of demos of unreleased products, such as efforts in robotics, Microsoft's online telescope, and Silverlight running on Windows Mobile (though that is still a year a way from being on the market, Watson said.) The company also showed off something it dubs a "replaceable PC,"--basically a combination of SoftGrid application virtualization and other technologies used to allow businesses to replace a worker's computer with new hardware and have them up and running in a few minutes' time.

Microsoft has a few more announcements scheduled for Wednesday, including a program designed to help fill the talent gap by connecting partners with the students who take part in the company's annual Imagine Cup.

And, as it is doing in many of its units, Microsoft is creating a "labs" component to its partner program. The first effort there is a social networking tool that will allow partners to collaborate and share ideas with one another. Microsoft is also announcing a new version of its online tool that connects customers with all of the thousands of partner-created tools in a vast database.

For years, Microsoft has maintained that the PC is the center of the digital home and office.

But Chief Software Architect Ray Ozzie said Tuesday that it's time for the company to acknowledge a new reality.

Ray Ozzie, Microsoft's chief software architect

(Credit: Microsoft)

"Over the past 10 years, the PC era has given way to an era in which the Web is at the center of our experiences--experiences delivered not just through the browser but also through many different devices including PCs, phones, media players, game consoles, set-top boxes and televisions, cars, and more," Ozzie said in a memo to be sent to employees on Wednesday (PDF).

In the memo, Ozzie outlines three principles to guide the company in this new era. Chief among those is the notion that "the Web is the hub of our social mesh and our device mesh." The notion of a Mesh is one that Ozzie has been working on for some time and has culminated in the Live Mesh service that Microsoft detailed on Tuesday.

He notes that this transformation has been a challenging one for Microsoft to embrace.

"More than two years ago when I wrote the memo entitled The Internet Services Disruption, much of the company was still focused on bringing our Office 2007 and Vista products to market," Ozzie said. "Aside from MSN, IE/IIS and our tools groups, it was truly 'software', not 'services', that was top of mind."

But, he said, "since then, we've made tremendous progress in our expansion toward 'software+services'--from the long-term quests we've undertaken and customer scenarios we now envision, to the great services and service-enhanced software we've begun to bring to market, and the amazing projects at various stages within our development pipeline."

In the memo Ozzie describes the software+services future as a merging of desktop, mobile and the Web data and devices, orchestrated through Live Mesh:

Office Live will bring Office to the web, and the web to Office. We will deliver new and expanded productivity experiences that build upon the device mesh vision to extend productivity scenarios seamlessly across the PC, the web, and mobile devices. Individuals will seamlessly enjoy the benefits of each - the rich, dynamic editing of the PC, the mobility of the phone, and the work-anywhere ubiquity of the web.

Ozzie indicates that the social graph--the relationships among people--will be part of what Live Mesh handles:

The web is first and foremost a mesh of people. Elements of this social mesh will be a first-class attribute of most all software and service experiences, as the "personal" of the PC meets the "inter-personal" of the web.

He stops short of specifically hooking Silverlight--Microsoft's cross-platform, cross-browser rich Internet application technology that competes with Adobe AIR-- into the Live Mesh experience, but the combination of the two brings Microsoft beyond its Windows-centric heritage.

(Credit: Microsoft)

The question for me is not how far Ozzie's thinking has evolved, but just how far his vision has spread within the company and where the pockets of resistance lie. In his latest memo he talks about this new world order affecting every area of the company, from Windows to developer tools to entertainment devices. That means he's going to need a whole lot of people to buy in to his philosophy for the vision to be made real.

Following is the full text of Ozzie's memo:

Over the past decade our lives, our businesses, and our society have been transformed by the web.

In its early days the web grew through the explosion of information portals as gateways to content, marketplaces for commerce, and communications tools such as email, IM and newsgroups that drove a sense of community on the internet. Over time, the significance of these "3 Cs" -- content, commerce, and community -- has expanded tremendously, growing in ways through which they've become intermixed and mutually reinforcing.

Content has changed at both the "head" and the "tail". The line between editorialized portals and blogs has blurred, and all are consumed through feeds. Beyond news, movies and music and television have all expanded to embrace the web. And the interrelation of content and community has created a world of "social media", where both head and tail content is intrinsically social by virtue of community linking, tagging, and ranking. Relationships and collective behavioral intelligence have changed how we stay informed, find and share media, and interact with one another.

Commerce on the web has moved well beyond the early online shopping cart. Nowadays, community is impacting commerce in dramatic ways. Head retailers such as Amazon utilize community extensively for recommendations, reviews, and wish lists. Tail commerce websites such as Craigslist utilize community extensively for conversation around local products. And Search has completely transformed online commerce. It's an essential utility for how we research, how we shop, and how we buy on the web. It's also become an essential mechanism for how we market on the web, and increasingly for how we sell on the web.

Community on the web once meant "group communications", largely through rudimentary tools such as email, IM and IRC, message boards and newsgroups. Today, the action has shifted toward using composite communications tools and platforms that mash together content, applications and commerce, all within the context of group interaction. These social platforms are altering the way we connect and coordinate, establish identity and affinities, and build reputation. While this notion of composite communications is most prominently demonstrated in how we use profile-centric consumer social networking tools, such as Facebook, the social platform is also finding its way into the workplace in the form of increasingly rich workspaces, both real-time and asynchronous, that integrate communications and content relevant to a project or a team.

As the "3 Cs" have evolved, so has the significance of online advertising as the economic engine powering our world of services. With growth projected from $40B today to $80B in the next three years, online advertising will continue to be the primary monetization mechanism for consumer services on the web. As advertising transitions more and more to being digital, measurable, and competitively bid, the "ad platform" is key. The advertising ecosystem surrounding this platform is reliant upon the continuous innovation of publishers and developers, whose interesting and engaging properties capture users' time and attention and ultimately serve to match advertisers with a relevant audience. Continuous innovation in such high-engagement products and services, in each area of the "3 Cs", will continue to provide the fuel to drive the advertising-based economic model.

Given this context, it's strategic that we invest broadly in solutions and partnerships that advance our position in current and future generations of content, community, commerce, and search, and also in an advertising platform that's attractive to advertisers, publishers and developers.

But while innovation in the "3 Cs", search and ads is essential for success in services targeting consumers on the web, their impact barely scratches the surface of the much broader effect that internet services innovation will have on individuals, businesses, and developers.

Indeed Microsoft's overall services strategy encompasses all of these areas: services for the individual, services for business, and services for developers. The intent of this memo is to map out that all-up strategy. I'll outline three principles that guide our work, and describe how those principles are woven into our myriad software and services offerings.

Central to this strategy is our embrace of both a world of the web and a world of devices. Over the past ten years, the PC era has given way to an era in which the web is at the center of our experiences - experiences delivered not just through the browser but also through many different devices including PCs, phones, media players, game consoles, set-top boxes and televisions, cars, and more.

It is our mission in this new era to create compelling, seamless experiences that combine the power of the internet, with the magic of software, across a world of devices.

Guiding Principles

There are three overarching principles guiding our services strategy - principles informing the design and development of products being implemented across all parts of Microsoft, for both individuals and business.

1. The Web is the Hub of our social mesh and our device mesh.

The web is first and foremost a mesh of people. Elements of this social mesh will be a first-class attribute of most all software and service experiences, as the "personal" of the PC meets the "inter-personal" of the web. Whether in work, play, or just life, the social element of software will continue to transform the ways that we interact with people with whom we have some affinity. All applications will grow to recognize and utilize the inherent group-forming aspects of their connection to the web, in ways that will become fundamental to our experiences. In scenarios ranging from productivity to media and entertainment, social mesh notions of linking, sharing, ranking and tagging will become as familiar as File, Edit and View.

We're also living in a world where the number and diversity of devices is on the rise; not just PCs and phones, but TVs, game consoles, digital picture frames, DVRs, media players, cameras and camcorders, home servers, home automation systems, our car's entertainment and navigation systems, and more. To individuals, the concept of "My Computer" will give way to the concept of a personal mesh of devices -- a means by which all of your devices are brought together, managed through the web, as a seamless whole. After identifying a device as being "yours", its configuration and personalization settings, its applications and their own settings, and the data it carries will be seamlessly available and synchronized across your mesh of devices. Whether for media, control or access, scenarios ranging from productivity to media and entertainment will be unified and enhanced by the concept of a device mesh.

2. The Power of "Choice" as business moves to embrace the cloud.

Most major enterprises are in the early stages of a significant infrastructural transition -- from the use of dedicated and sometimes very expensive application servers, to the use of virtualization and commodity hardware to consolidate those enterprise applications on computing and storage grids constructed within their data center. This trend will accelerate as enterprise applications are progressively re-factored from a centralized "scale up" model to the horizontal "scale out" requirements of this new utility computing model.

Driven in large part by the high-scale requirements of consumer services, the value of this utility computing model is most clearly evident in cloud-based internet services. By extension, cloud-based enterprise utility computing, infrastructure services, and enterprise applications are all becoming a reality, affording IT a range of new choices in how to deploy solutions across and between enterprises; within their own data center, in a partner's hosting facility, or with the vendor itself in the cloud. Software built explicitly to provide a significant level of server/service symmetry will enable IT to balance factors such as cost and control, and to leverage the skills of its key personnel most effectively. It will afford choice and flexibility in developing, operating, migrating and managing such systems in highly varied enterprise deployment environments that are distributed and federated between the enterprise data center and the internet cloud.

3. Small Pieces Loosely Joined for developers, within the cloud and across a world of devices.

Application design patterns at both the front- and back-end are transitioning toward being compositions and in some cases loose federations of cooperating systems, where standards and interoperability are essential. At the front-end, lightweight REST-based technologies have become ubiquitous, in some cases augmenting their WS-* counterparts, in integrating a broad variety of components combined seamlessly for the user at the surface of the browser. RSS and ATOM feeds have become lightweight channels and queues between software components. Declarative languages such as XAML have enabled rapid UI innovation and iteration.

At a higher level, myriad options exist for delivering applications to the user: The web browser, unique in its ubiquity; the PC, unique in how it brings together interactivity/experience, mobility and storage; the phone, unique in its extreme mobility. Developers will need to build applications that can be delivered seamlessly across a loosely coupled device mesh by utilizing a common set of tools, languages, runtimes and frameworks -- a common toolset that spans from the service in the cloud to enterprise server, and from the PC to the browser to the phone.

At the back-end, developers will need to contend with new programming models in the cloud. Whether running on an enterprise grid, or within the true utility computing environment of cloud-based infrastructure, the way a developer will write code, deploy it, debug it, and maintain it will be transformed. The cloud-based environment consists of vast arrays of commodity computers, with storage and the programs themselves being spread across those arrays for scale and redundancy, and loose coupling between the tiers. Independent developers and enterprises alike will move from "scale up" to "scale out" back-end design patterns, embracing this model for its cost, resiliency, flexible capacity, and geo-distribution.

Transformation of our Offerings

Successful experiences on the web are those that are organically compelling, highly engaging, and viral across their intended audience. By applying our three principles consistently across all the markets we serve, we have an opportunity to reshape our offerings for individuals, businesses, and developers, and to deliver a broad range of compelling scenarios.

CONNECTED DEVICES - We aspire to bring together Windows, Windows Live, and Windows Mobile by creating seamless experiences that span these offerings. Windows Live, for example, enables seamless communications and media experiences across Windows, Windows Mobile, and the Web. Live Mesh, a new services platform technology that will also become part of Windows Live, further extends the Windows / Windows Mobile / Windows Live experience by bringing your devices together to work in concert with one another using the web as a hub, enabling:

Unified Device Management - Users will register their devices through a simple, web-based service. Once a part of a user's device mesh, whenever they happen to connect to the internet the devices "report in" to the service -- e.g. for status, health, location, and to exchange/synchronize information. Mesh-aware device configuration/personalization will be done through the web, and full remote control of a device (e.g. remote desktop) will be available from anywhere.

Unified Application Management - Installation and management of "mesh-aware" applications on any or all devices, along with their application settings and data, will be simple and transparent for the user. Individuals will now enjoy the centralized cross-device purchase/deployment experience formerly available only within the enterprise environment.

Unified Data Management - Folders and files (e.g. documents and media) will be automatically synchronized and made available across any or all devices, as well as through the web. Because every folder can now have an extended web presence, even PC-based documents and media can now have a social mesh element to them.

CONNECTED ENTERTAINMENT - Building upon our device mesh vision, our aspiration is that individuals will only need to license media once, organize their subscriptions and collections once, and use any of their mesh-connected devices to access and enjoy their media - from the living room to the desktop to their pockets. And building upon our social mesh vision being interwoven into everything we do, each individual will be afforded a media-centric or gaming-centric web presence through which they can express their tastes/interests/affinities and interact with others through linking, sharing, ranking and tagging of music, video, photos, games, and more. This vision is being realized today through the Zune Social for media and Xbox LIVE for gaming. Services such as the MSN.com home page, MSN Mobile, MSN Video, Zune Marketplace and software such as Windows Mobile, Microsoft Mediaroom and Windows Media Center will be progressively transformed by this connected entertainment vision.

CONNECTED PRODUCTIVITY - Office Live will bring Office to the web, and the web to Office. We will deliver new and expanded productivity experiences that build upon the device mesh vision to extend productivity scenarios seamlessly across the PC, the web, and mobile devices. Individuals will seamlessly enjoy the benefits of each - the rich, dynamic editing of the PC, the mobility of the phone, and the work-anywhere ubiquity of the web. Office Live will also extend the PC-based Office into the social mesh, expanding the classic notion of "personal productivity" into the realm of the "inter-personal" through the linking, sharing and tagging of documents. Individuals will have a productivity centric web presence where they can work and productively interact with others. This broadly extended vision of Office is being realized today through Office Mobile and Office Live Workspace on the web, augmented by SharePoint, Exchange, and OCS for the connected enterprise.

CONNECTED BUSINESS - We will extend the benefits of high-scale cloud-based infrastructure and services to enterprises, in a way that gives them choice and flexibility in intermixing on-premises deployment, partner hosting, or cloud-based service delivery. Businesses large and small will benefit from services that make it easy to dynamically connect and collaborate with partners and customers, using the web to enable a business mesh. Business customers of all sizes will benefit from web-based business services. This vision is being realized today through the likes of Office Live Small Business. For enterprises, our new Microsoft Online Services provide managed, service-based infrastructure through offerings including SharePoint, Exchange, OCS, and Dynamics CRM. Our enterprise solution platform extends to the cloud through SQL Server Data Services, BizTalk Services, and many more services to come. At the lowest level within the enterprise data center, we-ve begun to deliver on our utility computing vision, with Windows Server 2008 and Hyper-V, and through our Systems Center products including Virtual Machine Manager.

CONNECTED DEVELOPMENT - As individuals embrace a world of devices and our device mesh vision, and as businesses embrace cloud-based services and server/service symmetry, developers will need platforms and tools that span seamlessly from cloud to server on the back-end, and from PC to browser to phone (and more) on the front-end. This vision is being realized today in our .NET family of runtimes including .NET Framework and Silverlight, supported by Expression Studio for designers and Visual Studio for developers, enabling developers to leverage their skills across all these environments. Our tools will be designed to support development of solutions that seamlessly incorporate multiple tiers, with some pieces on the PC, and others on the web or mobile; with some pieces on an enterprise server, and others running cloud-based utility computing infrastructure.

Transformation of our Company

As our industry has evolved because of this web-catalyzed services transformation, so too has Microsoft.

More than two years ago when I wrote the memo entitled The Internet Services Disruption, much of the company was still focused on bringing our Office 2007 and Vista products to market. Aside from MSN, IE/IIS and our tools groups, it was truly "software", not "services", that was top of mind.

Since then, we've made tremendous progress in our expansion toward "software+services" - from the long-term quests we've undertaken and customer scenarios we now envision, to the great services and service-enhanced software we've begun to bring to market, and the amazing projects at various stages within our development pipeline.

In light of all the work that we're doing, it's important that we build a shared sense of what Microsoft's path looks like in this transition toward software+services.

For consumers, advertisers and publishers, our investments in new forms of content, community, commerce, search and advertising are key. We're investing significantly to ensure tremendous audience engagement, and to provide an attractive and well-targetable audience, ensuring that we continue to be an attractive partner for advertisers and publishers within a vibrant and competitive advertising ecosystem.

For customers and partners who use and who've invested in Microsoft's myriad offerings, we feel there's tremendous growth potential in moving toward a world that fully embraces software, services and the web. The device mesh, the social mesh, cloud- based infrastructure, and server/service symmetry represent great opportunities across all markets we serve.

Over the course of this year, and progressively over the next few years, you'll see the principles and scenarios laid out in this document come to life through many new and service-enhanced products and services for individuals, businesses and developers. As you do, I hope you'll share my excitement and optimism as you experience how we're bringing together the power of the internet, with the magic of software, across a world of devices.

Ray Ozzie