Outside the Lines

On this week's EIC Squared podcast, ZDNet's Larry Dignan and I discuss this week's big stories. It was a busy week on the search front. Adobe is providing Google and Yahoo with Flash Player technology that allows their search engine crawlers to find and index SWF content, including Flash "gadgets" such as buttons or menus and self-contained Flash Web sites. It's good to make more information accessible via search engines. However, Microsoft has been silent on whether Live Search would index Flash content.

In addition, Microsoft bought Powerset for about $100 million to enhance its search platforms. It's not a substitute for acquiring market share via Yahoo Search, but it provides a foundation for making the search experience far more compelling and precise in fewer clicks.

Of course, the Microhoo drama continues this week with the latest rumors. Larry is ready for this opera to be finished.

Finally, we discuss a judge's ruling in Viacom's $1 billion copyright infringement suit against Google and YouTube.

U.S. District Judge Louis L. Stanton ruled that records of every video watched by YouTube users, including login names and IP addresses, should be given to Viacom's lawyers. Larry said it was like combining the worst aspects of a fishing expedition and a witch hunt. Viacom is maintaining that it won't look at personal data and Google is asking for time to anonymize the information. If Judge Stanton's ruling stands, the last shreds of personal ... Read more

As expected (see previous reports), Microsoft scooped up Powerset to buttress its search efforts.

Barney Pell, Powerset co-founder and CTO

(Credit: Dan Farber)

It's not a replacement for increasing market share by acquiring Yahoo Search, but it gives Microsoft some differentiated search technology and top engineers for less than $100 million. Ramez Naam, group program manager of Live Search, said the Powersoft negotiations happened in parallel with the Yahoo talks over the last few months. Google and Yahoo may also have been interested in Powerset, but no one is talking.

Whether Microsoft can leapfrog Google over the long term with this semantic engine remains to be seen.

Powerset had done a good job of creating a rich semantic layer on top of Wikipedia, but bringing natural language and slick semantic-based interfaces to the entire Web is a long-term and very costly endeavor.

"With an existing search infrastructure, incredible capital resources, unlimited data, a leading search team, and clear mission to revolutionize the search landscape, Microsoft can rapidly accelerate our progress in building semantic search technology and bringing it to full Web scale," Powerset's Mark Johnson said in a blog post about the acquisition.

Powerset can provide direct answers to queries from its Wikipedia and Freebase index and highlight the most relevant search results based on the meaning of the query.

According to a blog post from Satya Nadella, Microsoft's senior vice president of Search, Portal, and Advertising, Powerset's engineers will join the Search Relevance team and ... Read more

A few weeks ago I talked with Jonathan Heiliger, vice president of technical operations at Facebook, about the challenge of innovating quickly and building stable infrastructure while 250,000 new members are added to the social network every day. Check out the video on ZDNet.

Jonathan Heiliger

(Credit: CNET News)

Q: You've been at Facebook, I think, for about a year and it's been quite a ride I guess, scaling up from zero in 2004 to over 80 million today. How do you keep up with that hyper growth?
Heiliger: You're absolutely right--we've had a lot of growth. We add over 250,000 users every day, and that means a lot of infrastructure, a lot of servers, and constantly looking at new processes and looking at how we're doing things and ensuring that we're doing things the most efficient way possible, not just for delivering all the content to our users but to stay on top of what it costs to run the site.

How do you stay on top of the cost in terms of the kind of equipment you buy and how you work with the vendors? How do you prioritize those things?
Heiliger: One of the things we recently did was we ran an RFP process for the servers we buy from vendors and essentially did a bake-off with a number of different people looking at building servers on our own. What we concluded from that process was to continue to ... Read more

It's somewhat incomprehensible that Twitter has been unable to keep the service up and running. More than 10 years into the age of the Internet, with a huge amount of R&D publicly available about scaling Web applications, you would think that Twitter's engineers could figure it out.

A recent blog post from Twitter co-founder Biz Stone said help is on the way in the form of about $15 million in funding:

Twitter will become a sustainable business supported by a revenue model. However, our biggest opportunities will be worth pursuing only when we achieve our vision of Twitter as a global communication utility. To reach our goal, Twitter must be reliable and robust. Private funding gives us the runway we need to stay focused on the infrastructure that will help our business take flight. We will continue hiring systems engineers, operators, and architects, as well as consultants, scientists, and other professionals to help us realize our vision.

But the natives who love Twitter are getting restless, and they're losing faith. Twitter is chronically up and down, and key features, such as track and replies, disappear as company engineers try to save their patient from flatlining.

The father of RSS Dave Winer recently said, "Twitter, as it was conceived, was never meant to live."

"It's very possible with better engineering its architecture might have gone on for a few more years, but eventually it would have hit this wall, where there were too many people

Today is Bill Gates' last day in the office as a regular employee of the company he co-founded in 1975. But as non-executive chairman and someone who is deeply married to Microsoft, Gates is not disappearing from the company.

Chairman Bill Gates

(Credit: Dan Farber)

The transition has been well orchestrated, and he will still spend about 20 percent of his time working on Microsoft issues, such as the next-generation Office, natural interfaces, and search. And, he will still obsess and strategize about how to defeat Google.

Bill Gates field questions from this reporter.

(Credit: Michael Arrington)

I have been covering Microsoft and Bill Gates for the last 25 years, and I've had a few memorable run-ins with the him over that time. I remember asking him about upstart programming language Java's write once/run anywhere capability in an interview I did with him in the early 1990s. He sat forward in his chair and said with conviction that Java was a stupid idea. Behind that answer, the hyper-competitive Gates was thinking about how to slay the Java dragon. Several years later Microsoft C# appeared.

And who can forget his duel with David Boies in the U.S. Justice Department vs. Microsoft antitrust case. Gates believed that the government was out to destroy Microsoft, and went on the offensive. To this day, he chafes at being called a "convicted monopolist."

In many ways Gates is very much the same as when I met him a few decades ago. ... Read more

After attending GigaOM's Structure 08, I came away with a cloud-computing hangover. Just trying to define cloud computing is daunting given all the hype and companies thunderclapping.

Today the research firm Gartner has jumped on the cloud computing bandwagon, proclaiming that it "heralds an evolution of business that is no less influential than e-business," and defining it as massively scalable IT-related capabilities provided as a service using Internet technologies to multiple external customers.

Yahoo just announced a Cloud Computing & Data Infrastructure Group, which will develop computing infrastructure that balances scalability with cost effectiveness. What was Yahoo doing before it created this group?

I prefer the way Sun Chairman Scott McNealy talks about cloud computing. Ten years ago he was calling it the "big freakin' Webtone switch." Following is how he described it in December 2001:

That is the server, the storage, the operating system, the monitoring software, the clustering, the alternate pathing, multiple domaining, dynamic reconfiguration--and then it has a mail tone, a calendar tone, a news tone, an app server tone, and a directory tone. It has all of the different features of a big freaking WebTone switch and allows you to create this big jukebox. You can buy that all complete. Or you have one throat to choke and you can buy it all through a service provider that is SunTone certified. Or you can do what many IT directors do and they go out and buy the telephone switch by buying the chip from Intel,

SAN FRANCISCO--Speaking at the Structure 08 conference here, Sun Microsystems CTO Greg Papadopoulos predicted that by the beginning of 2010 the majority of systems sold would be for Web, high performance computing and software-as-a-service applications. "We are going through this phase change in computing in a big way," he said. He made a similar prediction last year.

Papadopoulos also advocated a free market in which all interfaces and formats are based on open standards; customers own their data, relationships, and metadata; and customers can extract, synchronize or purge their data unilaterally. This echoes recent efforts to promote openness and data portability.

Papadopoulos acknowledged that the nirvana of every customer or user in charge of their own data that lives in the cloud has challenges. Today, users cede control of their data to service providers like Google, Facebook, Microsoft, Yahoo, and others. It's not as easy for users to manage and move their data as it should be, which means users are generally stuck with the user experience and monetization schemes of the host sites. "It's proprietary systems all over again," Papadopoulos said. Over the last several years Sun has differentiated itself proprietary vendors, focusing on free open-source software and open standards.

Sun CTO Greg Papadopoulos

(Credit: Dan Farber)

Further out into the future, Papadopoulos expects that the technology infrastructure industry will be similar to the energy industry. In past presentations, he has called this transition the Red Shift.

Papadopoulos has predicted a "neutron star collapse of data centers," ... Read more

Speaking at Structure 08, Debra Chrapaty, corporate vice president of Global Foundation Services at Microsoft, shed some light on the cloud-based infrastructure supporting Microsoft's online services.

Despite characterizations that Microsoft is stuck in the client/server world, the company is spending billions to apply the cloud, or server/client, model, where most of the computing happens in the cloud and some small amount on the client (offline support for applications). But until Microsoft Office and other applications are built for the cloud, the laggard characterization will continue to stick to the company's forehead.

Debra Chrapaty, corporate vice president of Global Foundation Services at Microsoft.

(Credit: Dan Farber)

Microsoft has one of the biggest collections of Web sites, with 550 million users, 2 billion search queries, and 10 billion page views per month, as well as 8 billion messages on Microsoft Messenger per day. The company deploys 10,000 new servers per month on average to keep up with demand, Chrapaty said. She broke down Microsoft's model for building infrastructure into a three-letter acronym.

The cloud is all about GET--Growth, Efficiency, and Trust, Chrapaty said. In terms of growth, data centers are a $300 million to $500 million investment. "You have to make every kilowatt count," she said, noting that Microsoft has 35 criteria, such as network egress, power, and available staff, to determine locations for data centers.

Efficiency involves tools for manageability, operability, and sustainability, which translate into cost savings. "It's nice to go to Steve (... Read more

SAN FRANCISCO--During a panel discussion at the Structure conference here Wednesday, various representatives from the cloud-computing world offered their views. Panelists included:

• Christophe Bisciglia, senior software engineer, Google
• Jason Hoffman, founder and chief technology officer, Joyent
• Tony Lucas, CEO, XCalibre Communications
• Lew Moorman, senior vice president of strategy and corporate development, Rackspace
• Geva Perry, chief marketing officer, GigaSpaces
• Joe Weinman, VP of Strategic Solutions at AT&T

The panelists agreed that there will be open and proprietary, as well as specialized, cloud platforms. The discussion got a little heated between Google's Bisciglia and Joyent's Hoffman on the subject of open platforms and Google's BigTable software for distributed data storage.

"The question is, is it about selling your soul? You can't leave," Hoffman said during the panel, referring to Google's App Engine and cloud-computing platform. "There's been a lot published on what an open, loving cloud should do. We should give people real assurances that the cloud is a good place to be."

During the panel, Bisciglia said people can build a better mouse trap and compete with what Google offers. "When we publish something on BigTable, it is not to say that it is a lock-in, but it's our attempt to say that this is something that worked for us," he said.

"If your data is in Google's BigTable, you can't pull it out. You can't install it on your own hardware or leave. You have big brother telling ... Read more

In the early morning at Structure 08, AMR Research's Jonathan Yarmis described various tech trends around cloud computing. Mendel Rosenblum, a founder and technical lead behind VMware, outlined the role of virtualization in data centers.

Amazon CTO Werner Vogels

(Credit: Dan Farber)

Now Werner Vogels, vice president and CTO at Amazon.com, is talking about why Amazon is in the cloud computing business, how it got there, and why customers should want it. Instead of every company or developer doing the heavy lifting, dealing with the "muck" as Amazon CEO Jeff Bezos likes to say, Amazon opened up its software-as-a-service stack (Amazon Web Services) and infrastructure (Elastic Compute Cloud, S3, and SimpleDB) to external parties.

I've heard the Amazon story many times, but Vogels offered a few new tidbits, such as S3 is storing 18 billion objects and how Amazon thinks about building to its 1,000 services.

"Amazon built these services internally as tools, not as a framework. Each team can use whatever development tools they need. Infrastructure services need to be very generic and people can switch to competing services internally," Vogels said. For example, users could work with Amazon EC2 and a different storage service than S3.

Vogels outlined the core objectives and principles that cloud computing must meet to be successful: