Webware

It was inevitable that someone would seriously consider taking Google's dare.

Rupert Murdoch is reportedly thinking about removing all of News Corp.'s content from Google and striking an exclusive deal with Microsoft's Bing.

(Credit: Dan Farber/CNET)

For years, Google has all but dared traditional media companies trying to develop online businesses to live without the traffic it sends their way. The folks at the Googleplex make it clear that content owners who believe Google is unfairly indexing (or stealing, depending on your point of view) their content can easily remove that content from Google's massive corner of the Internet.

There's a tradeoff for that independence, of course: Don't expect the advertisers that have signed deals based on site traffic to pay the same amount next year.

News Corp. might be getting ready to do what many think is unthinkable. Reports have surfaced over the last several months, most recently in the Financial Times, that News Corp. is in talks with Microsoft to enact a plan that would see News Corp. properties hiding their content from Google's search engine in return for exclusive listing with Bing.

Rupert Murdoch, News Corp.'s famously cantankerous leader, isn't stupid: Microsoft would also have to pay News Corp. for the privilege of exclusive access to that content. But as Microsoft continues to lose billions of dollars a year on its online business, can it afford to be successful with this strategy?

Even if Microsoft is willing to cough up a huge sum (which Kara Swisher at Boomtown thinks is unlikely) for News Corp. content, this plan would only have a chance of turning the tables on Google if News Corp. and Microsoft can convince other large media companies to follow their lead.

First off, the practice of actually removing News Corp. content from Google would be relatively simple. News stories from The Wall Street Journal, commentary from The New York Post, and videos from News Corp.'s myriad cable and satellite television organizations can be tagged with a "noindex" tag, and Google won't index those pages as they are published. This also applies to pages that have been previously indexed, since they will be crawled again, this time with the new tag attached.

However, News Corp. would then need a backup plan to compensate for the revenue it would lose from the precipitous drop in traffic. With 65 percent of the search market, Google is the largest Web site in the world as measured by traffic. And its stated goal is to be the best information kiosk ever created by fielding queries and sending searchers on their way as fast as possible.

Murdoch has proposed removing his Web sites from Google only after constructing pay walls like the one used at the Wall Street Journal to limit free access to content, which is a somewhat controversial notion in this media era.

What News Corp. and Microsoft are reportedly discussing, however, is slightly different. Under the scenario outlined by the Financial Times, it does not appear that News Corp. would erect pay walls for all its content upon removal from Google. Instead, it would continue to make that ad-supported content available for free exclusively through Bing, helping offset the decline in traffic with a cash payment.

The two companies would then presumably market the hell out of the arrangement, because it would require a sizable shift in consumer expectations for Internet search. Right now, people are used to the idea that DirecTV is the only television provider that can offer a full package of NFL games every week, or that Comcast's Versus channel isn't available on DirecTV because of a licensing spat.

But that's not what they expect when they search online for news or information about a certain topic, and it would take some effort to educate them that The Wall Street Journal or Fox News' content can only be found if you're searching on Bing. Microsoft has already invested $100 million into Bing advertising, and would need to increase that amount to drive home the point that Bing is the only place you can find Fox News stories.

So will enough people be interested in that content as to change their search behavior and dramatically increase Microsoft's search market share? It's hard to see News Corp. moving the needle by itself, but modest results could embolden Microsoft to cut similar deals with other news companies and start the ball rolling toward the idea of Bing 2.0 as "the world's news search engine." That would be an interesting product.

As with just about everything, however, such a deal will likely come down to the amount Microsoft is willing to invest in such a project. Microsoft's Online Services Division, which runs Bing, is currently hemorrhaging money to the tune of $480 million in losses during its first quarter alone. Setting up content deals with the media industry would increase short-term costs with an iffy notion of when that investment would pay off in terms of increased search market share. And while Microsoft continues to milk Windows and Office profits, it can't throw money down a rabbit hole forever.

That means there's a sizable chance that this whole operation is geared around News Corp. negotiating a search and technology services deal with Microsoft to replace its current one with Google, which expires next June. Installing Bing as the search provider on News Corp. sites would generate increased searches for Microsoft while denying a common enemy Google some revenue, without kick-starting a huge battle that would have wide-ranging effects.

Murdoch has been able to tap into a well of frustration among those in the traditional media business over the way they are unable to duplicate the profits they enjoyed in the offline world on the Internet. But does he really want to call Google's bluff?

If so, he's banking on the notion that while basic news is a commodity, opinion and analysis is not. And whatever you might think of the various News Corp. properties, it's hard to argue they haven't earned a reputation for themselves as a unique source of opinion and analysis.

(Credit: Comscore)

Yahoo continues to lose share in the search market, as Google and Microsoft pick up the difference.

Comscore's measurement of the U.S. search market in October shows that Google--as usual--still dominates the search landscape. It now watches 65.4 percent of all searches pass through its servers, up 0.5 market share points from September of this year.

Yahoo, on the other hand, is going in the other direction as new friend Microsoft reaps the benefits. Yahoo lost 0.8 market share points in October compared to September, now down to 18 percent of the market. It has been in a steady decline this year, as Microsoft has gained share with the relaunch of Bing: Microsoft almost cracked the 10 percent barrier in October with 9.9 percent of all searches, gaining 0.5 percentage points compared to September.

Total searches grew 3 percent from September to October. Searches done through Google increased 5 percent and those done through Microsoft increased 8 percent. Yahoo searches decreased by 1 percent over the same period.

Unlike when you stand over your coworker's desk, Microsoft's Bing search engine actually works better when you hover.

One of the key features of the would-be rival to Google is that when you hover to the right of a result, you can get a preview of what to expect. As part of an update this week, Bing's hover result will now feature more information including a thumbnail preview of the site in question.

Microsoft is using Wolfram Alpha to help power certain results, such as this search for the fat content of french fries.

(Credit: CNET News)

One of the ongoing challenges for Bing, besides just getting more people to use the site, is letting them know that the hover feature is there. Microsoft's research has shown it gets high usage from those who know about it, but also finds that lots of people don't know the feature is there. Microsoft has been experimenting with some different visual cues that might make it easier to stumble upon the previews.

The hover feature was developed by the San Francisco-based team that Microsoft acquired as part of last year's acquisition of Powerset. Powerset, which developed a semantic search technology, also powers Bing's index of Wikipedia.

Bing's fall update update also includes the first fruits of a deal with Wolfram Alpha. As part of that arrangement, certain health related searches, such as "how many calories in a hamburger" will now feature information from Alpha. Bing will also rely on Alpha for some math calculations, Microsoft said in a blog posting on Wednesday. Wolfram noted that Microsoft is one of the first customers for a commercial licensing program that was formally announced several weeks ago.

Other changes to Bing include improved local results for topics such as weather and events.

It's all part of a wave of updates Microsoft is making to Bing this week. On Tuesday, Microsoft said it is moving its MSN Video site under the Bing umbrella, with a new video page that can be used to watch videos from places like Hulu and elsewhere.

The company also announced some enhancements to Bing Maps, including the ability to use the mouse to alter a suggested route and have one's directions re-calculated.

The improvements come as Microsoft is looking for ways to stand out from Google as it tries to wrest share from its much larger rival. The software maker has seen a modest uptick but faces steep hurdles in trying to make more significant gains.

Experian Hitwise said Wednesday that Bing's share reached 9.57 percent in October. That's up from 8.96 percent in September, but still well behind Google, which had more than 70 percent and Yahoo, with 16 percent of the U.S. search query market.

While adding features is clearly important, trying to stay ahead in the search game can be quite a challenge. Just hours after Microsoft announced a deal last month to index real-time tweets from Twitter, Google announced plans to do the same.

Microsoft has also gotten some unwanted attention for one of its features--the Bing Cashback program--where users can get a portion of their online transactions rebated by starting off on Bing. A blog posting outlined a flaw in the mechanism that could allow people to get cash back without ever spending money via Bing.

That posting was pulled after a demand from Microsoft's lawyers.

We all know the mathematical adage detailing how many words a picture is worth. The principal applies equally to search results. Search for anything in Google, Bing, and Yahoo and see how long it takes your eyes and brain to max out on all the written input. (The concept of text fatigue also applies to blog posts, which is why we've included a nice, large picture near the top of ours.)

(Credit: Screenshot by Jessica Dolcourt/CNET)

If you browse the Web with Firefox (Windows | Mac), the free extension SearchPreview (Windows | Mac) breaks up text blocks by inserting thumbnail images of the site's homepage to the left of the text, where your eye naturally goes. If you're unsure which of the many returned links you really want, a glance at the thumbnails could settle the dispute.

If the thumbnail image scenario sounds familiar, that's because until recently SearchPreview has been known as GooglePreview. An upgrade to version 4.0 added support for Microsoft's Bing search engine and repositioned the product with a name change.

As you use SearchPreview, you may notice some results tagged as SearchPreview's sponsored links. These are the developer's way of recouping costs, but you can disable the sponsored results from the Options menu of the add-ons manager.

(Credit: Microsoft)

Just days after Microsoft cranked out a touch-optimized Bing search page for touch screen phones comes another announcement for mobile searchers. Microsoft on Monday reported that Verizon will roll out the first Bing mobile app to BlackBerry Storm 1 users. The Bing mobile app already comes preloaded on the BlackBerry Storm 2.

After Verizon pushes Bing for mobile, Storm owners should be able to click the icon in the program list to get started.

In addition to search are features carried over from a former incarnation of Microsoft's search app, Windows Live for mobile, such as voice search, and a map that includes driving directions, traffic details, and a location feature to search your whereabouts. There's also a way to save favorite searches, and to quickly get at local searches.

We first encountered a variation of the Bing app on Windows Mobile 6.5 phones. While many features are nearly identical to the Windows Live mobile app, the refreshed search engine should produce more accurate results.

Bing's touch-friendly mobile interface lets you browse movies with your finger.

(Credit: CNET)

Microsoft on Friday released a refreshed version of its mobile search site (m.bing.com) that's optimized for touch-screen devices. The new page includes finger-friendly buttons that are easier to both identify and to press, as well as a movie finder that lets users browse by what's near them by time and theater.

So far, the only devices that work with it are the iPhone/iPod Touch, T-Mobile G1, Samsung Omnia, Verizon Imagio, and the Zune HD. Microsoft says support for other phones and portables is coming. In the meantime, phones that can't access the touch-friendly interface get defaulted to a simpler version.

Also worth noting is that the touch interface is only available to users in the U.S. for the time being.

Along with touch, Microsoft also added two new search query types that pull from near real-time data sources. This includes a way for users to check on NFL football scores and flight status. Users looking to get an updated score or player stats just need type in the team or player name. As for flights, you'll need the airline and flight number and it will cull the most recent information about arrivals, departures, or delays.

Now how about fitting some of that neato visual search action on the mobile site too?

Updated 4:30 p.m. PDT with additional details from Google.

It was indeed a nonexclusive deal: Google is going to be indexing real-time Twitter messages in search results, in a deal announced just hours after Microsoft debuted integration of "tweets" into its own search engine, Bing.

A post on the official Google blog by Vice President of Search Marissa Mayer explained it: "We believe that our search results and user experience will greatly benefit from the inclusion of this up-to-the-minute data, and we look forward to having a product that showcases how tweets can make search better in the coming months," the post read. "That way, the next time you search for something that can be aided by a real-time observation, say, snow conditions at your favorite ski resort, you'll find tweets from other users who are there and sharing the latest and greatest information."

Google has "reached an agreement," but the search results have not gone live like Microsoft's have on Bing. Reports started to surface earlier this month that Twitter was in separate talks with both Google and Microsoft--which also has a deal with Facebook that will be launching down the road.

Google plans to turn on the service "soon," said Johnna Wright, product manager for Google Search, declining to provide further details. The company has been working on this "complicated" problem for some time, she said; Mayer said earlier this year that microblogging search was a priority for Google in 2009.

It's just way too difficult to manually crawl Twitter for tweets, said Jack Menzel, group product manager for Google Search. Google would have to bombard Twitter's servers constantly via its public API, and the result wouldn't be pretty for anyone. So, instead Google and Twitter have cut a deal where Google is essentially licensing a data feed from Twitter to get that information in search results.

How will it be presented? Google isn't ready to talk about that yet in detail, but Wright said tweets would be presented within regular search results. "Relevancy is paramount," Menzel said, but it's also tricky: sometimes you might want the result from the guy with only 30 followers who knows what's happening on a given street corner, sometimes you might want the industry expert's quick take on a product announcement.

So will Facebook strike a Google deal, too? Onstage at the Web 2.0 Summit in San Francisco on Wednesday afternoon, Facebook Chief Operating Officer Sheryl Sandberg said that Facebook has "nothing to announce" regarding rumors of a search deal with Google.

Google wants to work with lots of different companies that are providing this type of information, Menzel said, although he declined to comment on any specific company. Following Google's announcement regarding Twitter, it announced a Google Labs product called Social Search for organizing streams of status updates and news feeds.

Bing's Twitter search starts with a zeitgeist view.

(Credit: Screenshot by Rafe Needleman/CNET)

Microsoft is getting into the real-time search business, as we reported earlier Wednesday from the Web 2.0 Summit. It's good to see a mainstream product dive into this stream, as one of the big issues with searching Twitter is that timeliness can swamp relevancy.

Bing has the opportunity to leverage its well-developed search engine chops to address this--not only will public tweets will show up in search results, Bing can rank results based on relevance of the post, the popularity of the writer, and other, more complex factors.

Uncharacteristically for Microsoft, the new search feature went live shortly after the announcement. (We're told the Facebook integration, which was also announced, will be rolled out in the future.) Here's how Twitterized Bing works for users so far:

The main page give a nice overview of trending topics with a search cloud at the top of the page and a list of popular links that are being shared below it. It's a good way to get a sense of the buzz on Twitter at any moment.

Bing's basic Twitter search result page gives you both live tweets (at the top), and shared links (below).

(Credit: Screenshot by Rafe Needleman/CNET)

Search results pages themselves are likewise split into two sections, a live feed at the top with just four tweets, and a list of shared links at the bottom. Results stream in live at the top of the page, but you can pause the influx.

The "Best match" search juggles the display order to put relevant tweets up top, even if they're not the most timely.

(Credit: Screenshot by Rafe Needleman/CNET)

If you click on the link to "see more tweets" on the main result page, you get a full page of tweets on your query, with the interesting option to sort the results by "Best match." If you choose this, Bing takes a stab at ranking results based on their content and possibly other factors, like popularity and online status of the writer.

Timeliness is still a factor in "Best match" results, so you won't get day-old tweets at the top of the list on a hot topic, but adding a relevancy sort on top of that does make the search results more useful. This is especially true for hot topics where tweets feeding into a time-only sort can end up pushing useful and relevant content right off the page.

Bing unpacks short URLs to show you what people are sharing, with no surpise links.

(Credit: Screenshot by Rafe Needleman/CNET)

Back on the main result page, there are links related to your search query. These are automatically unpacked from URL shorteners like Bitly. The link results have under them tweets that included a short link to the page, even if different shorteners were used to get there. Bing's Twitter search thus does a good job of pulling commentary together on a topic (a link) even from people who've never communicated with each other on the service.

None of what Bing does with Twitter is startlingly new. Twitter's own search gives great real-time Twitter results. Other engines like Twazzup and Scoopler combine relevancy rankings into their results. And OneRiot does a very good job with shared links. But it is good to see real-time content start to bleed into mainstream search. It could be useful and relevant for everyone.

But this story won't get truly interesting until the real-time feeds, from Twitter and elsewhere, start to infect the mainstream Web search results. When a trending topic or popular shared link on Twitter starts to change the way standard results are ranked, we'll start to have truly real-time search for all content. Twitter will have an impact across the Web, even for people who never use it.

(Credit: Bing)

SAN FRANCISCO--Microsoft is indeed bringing real-time search results from Facebook and Twitter to its Bing search engine thanks to two partnerships, search head Qi Lu and senior vice president Yusuf Mehdi announced at the Web 2.0 Summit here on Wednesday.

The Twitter partnership, which will bring all real-time public tweets to Bing, went live in beta on Wednesday at Bing.com/twitter. The Facebook deal, which will access all information shared publicly on the social network, will arrive "at a later date," Mehdi said. It's all part of Bing's strategy to harness "the emerging hot area of real-time information," he added.

No financial terms were disclosed for either deal; in a talk later on Wednesday at Web 2.0 Summit, Facebook chief operating officer Sheryl Sandberg said that "no money changed hands" in its deal with Microsoft.

"You have to do more visual things, you have to do more sophisticated things, and you have to have better access to data," Mehdi said.

This will also give Microsoft an edge on Google, which currently has no such deals in place--but both Facebook, in which Microsoft has a financial stake, and Twitter are rumored to also be talking to Google as well. (Editor's note: Separately, Google announced a similar deal with Twitter on Wednesday.)

The Twitter deal is nonexclusive, Lu said. He did not comment on Facebook.

Users of the Bing Twitter search can see tweets that match a search query listed in pure chronological order, much like Twitter's own search engine (which it built in with the acquisition of a third-party app called Summize). But you also have the option to see search results in a mode that Mehdi called "best match," where he explained that Bing's team will "apply a bunch of our search techniques and relevancy to improve the results." This will mean that popular tweets and heavily retweeted tweets are brought higher up in results, and spam and duplicate results (as well as adult content) will be filtered out.

The Bing Twitter search also aggregates information around hot topics that goes far deeper than Twitter's "trending topics": the most popular links shared on a given topic, for example, and a way to see where shortened URLs redirect. (R.I.P., Rickroll.)

The 'top links' feature in Bing's Twitter search.

(Credit: Bing)

It's still a work in progress, largely because Facebook and Twitter themselves are changing rapidly as well. "Facebook and Twitter, especially Twitter, are really emerging communication platforms, just a lot of growth and the dominant usage patterns...are still evolving in many ways," said Lu, who recently made the jump from Yahoo to Microsoft.

This post was expanded at 2:54 p.m. PDT.