Outside the Lines

In the early morning at Structure 08, AMR Research's Jonathan Yarmis described various tech trends around cloud computing. Mendel Rosenblum, a founder and technical lead behind VMware, outlined the role of virtualization in data centers.

Amazon CTO Werner Vogels

(Credit: Dan Farber)

Now Werner Vogels, vice president and CTO at Amazon.com, is talking about why Amazon is in the cloud computing business, how it got there, and why customers should want it. Instead of every company or developer doing the heavy lifting, dealing with the "muck" as Amazon CEO Jeff Bezos likes to say, Amazon opened up its software-as-a-service stack (Amazon Web Services) and infrastructure (Elastic Compute Cloud, S3, and SimpleDB) to external parties.

I've heard the Amazon story many times, but Vogels offered a few new tidbits, such as S3 is storing 18 billion objects and how Amazon thinks about building to its 1,000 services.

"Amazon built these services internally as tools, not as a framework. Each team can use whatever development tools they need. Infrastructure services need to be very generic and people can switch to competing services internally," Vogels said. For example, users could work with Amazon EC2 and a different storage service than S3.

Vogels outlined the core objectives and principles that cloud computing must meet to be successful:

Sanford C. Bernstein analyst Jeffrey Lindsay is betting that Google and Amazon.com will loom as the two giants of Internet. According to Lindsay's report, "U.S. Internet: The End of the Beginning," cited by Reuters, "Both Google and Amazon.com are still racking up annual growth rates in the 30-40 percent range, with only a relatively modest slowdown in sight."

Given how those two companies own their respective fields, it's not a stretch to forecast them as long-term winners in the coming years. Google and Amazon have done the best job of creating clear value propositions for online users in the last decade, although Google has the better margin business selling ads on search pages.

Google owns search, with a nearly 70 percent share, and is moving into the applications space; Amazon is the premier online, personalized retail shopping site; and both are poised to become major providers of computing infrastructure services for the planet

Google and Amazon have momentum and traction, but that doesn't mean they are invulnerable. They have only a decade of history, and they would acknowledge that they could be knocked off, just as they knocked off a variety of competitors on their road to greatness.

A scenario in which Google or Amazon are taken down isn't likely in the next several years, however. The two are well established and the online market is reaching early adolescence, making it more difficult for newcomers to be supremely disruptive to incumbents. If competitors become a nuisance, Google and Amazon have the clout and deep pockets to acquire or extinguish them, if they see them coming.

The two big winners in Lindsay's report could take some lessons from Microsoft if they want to stay on top. Microsoft can provide examples of how to diversify and grow, as well as cautionary tales about treading on antitrust laws, getting big and missing major shifts, such as the initial phase of the Internet. Having the smartest people in house has helped Microsoft and the newer giants succeed, but ultimately they have blind spots. They would do well to heed the advice of former Intel executive Andy Grove: Only the paranoid survive.

CARLSBAD, Calif.--Amazon.com founder and CEO Jeff Bezos kicked off the morning proceedings here at D6 after a night of polite carousing by industry luminaries. During the interview with D co-host Walt Mossberg, Bezos announced a streaming-video service and explained his foray into hardware with the Kindle e-book reader.

On the subject of video and music delivery, Bezos said, "We are working on a new version of video-on-demand, a for-pay streaming service we will release in the next couple of weeks. The streaming service will start instantly, and it's a la carte, for pay."

Regarding competing with Apple's iTunes services, Bezos said it is clearly in the self-interest of music companies to have competitors.

Walt Mossberg asks Jeff Bezos about the Amazon Kindle.

(Credit: Dan Farber/CNET News.com)

The Kindle is clearly a passion for Bezos. It follows on his love for selling books online, which was the origin of Amazon, and developing a new market for digital content delivered over wireless networks.

"We base our strategy on customer needs instead of what our skills are...Customers will eventually need things that you don't have skills for, so (you) need to renew yourself with new skills," Bezos said. If Amazon doesn't extend into new product categories, the company will get outmoded, he said.

Bezos wouldn't disclose Kindle sales. "On a title-by-title basis, with 125,000 titles for Kindle, and you look at Amazon's physical sale of the same books, Kindle sales are more than 6 percent of the universe of 125,000 titles," he said. Amazon reduced the price of the $399 Kindle by 10 percent this week.

While Bezos said he was happy with the sales of the Kindle, the price cut and the heavy promotion of the device on Amazon's site could mean sales aren't spectacular. The Kindle could be a meaningful financial component in Amazon's business, Bezos said, but he didn't put a figure on the Kindle's contribution to annual revenue.

Regarding the fate of physical books, Bezos said the vast majority of books will be read electronically. Just as horses haven't gone away, books will be around, he quipped. "We see Kindle as an effort to improve the book, even though it hasn't changed in 500 years," he added.

"You can't ever outbook the book, so you have to do things that you can't do with a book, such as in-stream dictionary lookup, changing fonts, and wireless delivery of content in 60 seconds," Bezos said. "We have to build something better than a physical book."

Bezos said he did research into the smell of the book--glue, ink, and mildew. "We can never capture that," he said, adding that the container is not important; the narrative is. He wants to make long-form reading more frictionless so that people read more.

Mossberg asked Bezos about adding new features to the Kindle and its utility as a Web browser.

"There are things that fit into the Kindle form factor and don't interfere with the purpose of the device. But the device is not a cell phone or bunch of things. It should be able to browse the Web," he said. "If you were trying to build the perfect Web-browsing device, you wouldn't use electronic ink. It's not the right display technology for high-quality Web browsing."

"You might say the Web is the most important book in the world," he added, but that's not something the Kindle is designed to read as well as other devices.

Click here for full coverage of the D: All Things Digital conference.

Evri has a new twist on content navigation and discovery. Debuting Wednesday at D6, Evri is not a search engine, according to CEO Neil Roseman, but a "data graph of the Web" that leads to "incremental content engagement."

"What doesn't work well is when you get to other places on the Web," Roseman told me. "We read sentences, extracting the subject, objects and verbs, and map to other content on the Web." Evri uses entity extraction, natural language processing, statistical analysis, and other technologies to create relevant connections based on meaning and concepts without human intervention.

Evri creates profile pages, which are like search results, that include a variety of lenses for an entity, such as top connections (entities most closely associated with the target entity), people, location, products, organizations, and events.

Evri profile pages show five top connections as a starting point for drilling down into the related content and concepts.

The profile pages are somewhat like what you get from Mahalo, which is human-powered, but closer to Powerset, Hakia, Twine, and and other new services that leverage semantic and natural language processing technologies to map concepts and meaning rather than keywords.

Evri also is planning to offer content publishers widgets that produce related content for a particular page, similar to what Sphere (recently acquired by AOL), Inform, and Aggregate Knowledge provide.

Roseman is focusing Evri as a consumer product. He spent 10 years at Amazon working on several projects, including searching inside books, the MP3 store, and the server side of the Kindle reader. Currently, Evri has parsed less than 1 percent of the Web, working with 20,000 to 30,000 top-level domains and some full-text providers. "Once we distribute the widget to content providers, we will incrementally add more to our deep parsing, and figure out what drives the most page views on a daily basis and build the network over time," Roseman said.

Evri is expected to go into beta testing in a month, Roseman said. Some of the processing will be done via Amazon's Elastic Compute Cloud facility. He noted that scaling to cover more of the Web is very hard. Evri will be ad-supported and will not charge content partners. "We will give partners all the revenue," Roseman said. "We want to build the network and get people to use Evri."

Seattle-based Evri has 36 people, mostly engineers, and is wholly funded by Vulcan Capital. So far Vulcan has poured about $8 million into the company, including the acquisition of some technology and engineering talent from Insightful, Roseman said. The company plans to go for Series A funding round this year.

Click here for full coverage of the D: All Things Digital conference.

With the platform-as-a-service revolution getting into full swing, developers (especially in start-ups) have more options for creating and deploying applications without the hassle and more extreme cost of setting up and maintaining infrastructure.

Dion Hinchcliffe at ZDNet compares Amazon's approach to providing infrastructure services to Google's. He found that Amazon's set of services is more flexible but not as integrated as Google's App Engine.

(Credit: Dion Hinchcliffe, ZDNet)

What if you realized that you didn't want to host your application on Google App Engine anymore? Good luck; almost everything you are given access to is proprietary--that means all your data is locked into BigTable in a format that isn't like a traditional relational database. It's also very tempting to use the APIs Google provides to interface with things like Google accounts.

On top of that, you will be using the "Webapp framework" that Google built that makes writing Python applications really nice--but good luck porting that to another language or putting it on a machine of your own.

On the other hand, Google is just trickling out its platform-as-a-service with support for Python. Support for other languages will follow. Whether Google would support other databases in its cloud remains to be seen.

On the road to the elusive Web 3.0 (something to do with semantics, meaning, and context rather than just data, links, and AJAX), core infrastructure is beginning to move from the edge to a center inhabited by companies such as Amazon, Salesforce.com, Joyent, and now Google with its new App Engine.

Call it Web 2.5, where the platform-as-a-service providers allow developers to create Web applications via the cloud and for users to consume them on any Web-connected device, anytime and anywhere. It eliminates what Amazon's Jeff Bezos describes as the "muck," the undifferentiated heavy lifting, such as setting up and maintaining servers, databases, storage, and networks.

It also leverages data centers from large players like Amazon and Google that were built from the ground up to support Web applications at huge, virtualized scale and with high reliability and relatively low cost. And, it creates potentially giant subscription-based revenue streams for the platform-as-a-service providers. They become utilities providing Web services to the planet and managing the high-value personal profile data.

Google App Engine, which was unveiled tonight at Google's Mountain View, Calif., headquarters, offers similar capabilities to Amazon's EC2, S3, and SimpleDB services. Google App Engine is limited to using the Python language, Google APIs, and a relatively modest amount of storage, compute cycles, and bandwidth per day currently, but you can see where this is heading.

Google could parlay its search and advertising technology, market dominance, and its infrastructure prowess into a powerful engine that runs and monetizes thousands or millions of externally developed applications.

Salesforce.com provides a more mature example today with its Force.com platform. It allows developers to write applications, mostly CRM-oriented, in a variety of languages that can run natively on the Salesforce.com software platform and data centers.

(Credit: salesforce.com)

In many ways it is the Microsoft model--you need a subscription (license in the old days) to the platform to run your application. In this case, "run" means that Salesforce.com provides developers all the software and hardware services in exchange for a fee, which is based on specific metrics, such as Web services calls.

Rival NetSuite, as well as smaller outfits such as Bungee Labs, are seizing on the concept of providing complete cloud-based development and deployment platform services.

Microsoft hasn't yet shown its cards in the platform-as-a-service arena. Nor has the object of its affection, Yahoo. Microsoft has talked about SQL Server Data Services and the grand synchronization mesh, but it hasn't revealed any plans for an end-to-end hosted platform-as-a-service for developing and serving applications from the cloud. Mary Jo Foley has some insight on that topic.

Web 3.0 as envisioned by Tim Berners-Lee is not around the corner, but it is busily percolating. In parallel, platform-as-a-service is evolving, the plateau of Web 2.5. When the two meet, Web 3.0 will have arrived.

TechCrunch is speculating that Google may begin the first major phase of becoming an infrastructure provider for developers by exposing its BigTable data storage system as a Web service. This service would be similar to Amazon's SimpleDB service, which automatically indexes data and provides an API for storage and access.

I've queried Google on this potential news and await a response. Given Google's prowess at delivering applications from the cloud, it's logical to expect the company to become a platform for application services, with APIs for storage, compute cycles and databases--similar to what Amazon has done with its S3 storage and Elastic Compute Cloud along with SimpleDB.

It's a way for Google to leverage its massive infrastructure build out, with hundreds of thousands of custom servers running in parallel, and deep computer science expertise by effectively becoming the network service for the planet.

It's the "Red Shift," utility computing concept offered by Sun CTO Greg Papadopoulus. He is predicting a "neutron star collapse of datacenters."

At some point, businesses won't build their own datacenters and developers will program on the network itself. Google, Sun and a few other megaliths will provide the computing resources with brutal efficiency for utilization, power, security, service levels and rapid idea-to-deploy time, Papadopoulus said. It's a model that salesforce.com has adopted on a smaller scale with its platform-as-a-service.

Google describes, BigTable as a "distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers."

The API includes functions for creating and deleting tables and columns, as well as for changing cluster, table, and column family metadata, such as access control rights, according to a white paper on BigTable, which gives the following description of its evolution and usage:

Over the last two and a half years we have designed, implemented, and deployed a distributed storage system for managing structured data at Google called BigTable. BigTable is designed to reliably scale to petabytes of data and thousands of machines. BigTable has achieved several goals: wide applicability, scalability, high per- formance, and high availability. BigTable is used by more than sixty Google products and projects, includ- ing Google Analytics, Google Finance, Orkut, Person- alized Search, Writely, and Google Earth. These prod- ucts use BigTable for a variety of demanding workloads, which range from throughput-oriented batch-processing jobs to latency-sensitive serving of data to end users.

The BigTable clusters used by these products span a wide range of configurations, from a handful to thousands of servers, and store up to several hundred terabytes of data. In many ways, BigTable resembles a database: it shares many implementation strategies with databases. Paral- lel databases [14] and main-memory databases [13] have achieved scalability and high performance, but BigTable provides a different interface than such systems.

BigTable does not support a full relational data model; instead, it provides clients with a simple data model that supports dynamic control over data layout and format, and al- lows clients to reason about the locality properties of the data represented in the underlying storage. Data is in- dexed using row and column names that can be arbitrary strings. BigTable also treats data as uninterpreted strings, although clients often serialize various forms of struc- tured and semi-structured data into these strings. Clients can control the locality of their data through careful choices in their schemas. Finally, BigTable schema parameters let clients dynamically control whether to serve data out of memory or from disk.

Illustration from a Google white paper, BigTable: A Distributed Storage System for Structured Data

(Credit: Google)