Webware

Vic Gundotra, Google's vice president of engineering, takes a photo of the Itsukushima Shrine in Japan. The Google Goggles feature successfully identified it.

(Credit: Stephen Shankland/CNET)

MOUNTAIN VIEW, Calif.--Google's first search engine let people search by typing text onto a Web page. Next came queries spoken over the phone. On Monday, Google announced the ability to perform an Internet search by submitting a photograph.

The experimental search-by-sight feature, called Google Goggles, has a database of billions of images that informs its analysis of what's been uploaded, said Vic Gundotra, Google's vice president of engineering. It can recognize books, album covers, artwork, landmarks, places, logos, and more.

"It is our goal to be able to identify any image," he said. "It represents our earliest efforts in the field of computer vision. You can take a picture of an item, use that picture of whatever you take as the query."

However, the feature is still in Google Labs to deal with the "nascent nature of computer vision" and with the service's present shortcomings. "Google Goggles works well on certain types of objects in certain categories," he said.

Vic Gundotra, Google's vice president of engineering, speaking at a Google search event Dec. 7 in Mountain View, Calif.

(Credit: Stephen Shankland/CNET)

Google Goggles was one of the big announcements at an event at the Computer History Museum here to tout the future of Google search. The company also showed off real-time search results and translation of a spoken phrase from English to Spanish using a mobile phone.

"It could be we are really at the cusp of an entirely new computing era," Gundotra said, with "devices that can understand our own speech, help us understand others, and augment our own sight by helping us see further."

Offering one real-world example of the service in action, Gundotra said that when a guest came by for dinner, he snapped a photo of a wine bottle she gave him to assess its merits. The result--"hints of apricot and hibiscus blossom"--went far beyond his expertise, but that didn't stop him from sharing the opinion over dinner.

He also demonstrated Google Goggles to take a photo of the Itsukushima Shrine in Japan, a landmark tourists may recognize even if they can't read Japanese. The uploaded photo returned a description of the shrine on his mobile phone.

Although the service can recognize faces, since faces are among the billions of images in the database, it doesn't right now, Gundotra said.

"For this product, we made the decision not to do facial recognition," Gundotra said. "We still want to work on the issues of user opt-in and control. We have the technology to do the underlying face recognition, but we decided to delay that until safeguards are in place."

Google's search is a near-constant work in progress as the company strives to grow beyond supplying search results in the form of 10 hyperlinks to various Web pages.

"It's not just about 10 blue links," said Marissa Mayer, Google's vice president of search and user experience. "It's about the best answers."

"In the past 67 days, we launched 33 different search innovations," she boasted. "That's one innovation every two days."

Three more in the pipeline came to light on Monday. First, the mobile version of Google's search service to suggest completions to search queries now is geographically smart. That means, for example, a person in Boston typing "re" in a search box will see "Red Sox" as a suggested completion but a person in San Francisco will see "REI."

'Near me now' is a mobile service that shows local services to a mobile search user.

(Credit: Stephen Shankland/CNET)

Second, a "near me now" service due to launch in coming weeks can tell users of iPhones and Android devices what's near them at a particular moment. Third, location supplied by the mobile phone can adjust product search results to show nearby stores that have a particular item in stock.

Sci-fi vision
Google isn't afraid of raising expectations of the service to the sci-fi level, where concepts such as augmented reality--an overlay of computer data that supplements what people see in the real world--have flourished for years.

Eventually, Google wants a system that lets people point to an object and retrieve information on it, Gundotra said--turning a person's finger into a real-world mouse pointer. "Today marks the beginning of that visual search journey," Gundotra said.

Google's system, like its Picasa face recognition software for photo management and face blurring in Google Maps' Street View, employs technology stemming from Google's 2006 acquisition of Neven Vision, a start-up focusing on face and object recognition. Founder Hartmut Neven, still a Google employee, was at Monday's event.

Neven expressed pride for one aspect of the system: the fact that much of its background work happens with no human interaction through a process he called "unsupervised learning."

"The algorithms build models for visual recognition are unsupervised," Neven said. "Based on the photos we find, models--for example, the Empire State building--will emerge."

Live translation
Speaking of science fiction, Google also showed off technology that could turn mobile phones into a computerized translation system. It wasn't quite the babelfish of "The Hitchhiker's Guide to the Galaxy," but it did translate Gundotra's question about where the nearest hospital is located into Spanish.

Marissa Mayer, Google's vice president of search and user experience, speaks at a Google search event Monday.

(Credit: Stephen Shankland/CNET)

The technology works using a new communications conduit to Google servers. The raw utterance recorded by the mobile phone is sent to Google's servers, which first interpret it as English. It's then translated into Spanish, and the text is sent back to the mobile phone. A text-to-speech synthesizer on the phone--for the demonstration, a Droid model running Google's Android operating system--reads out the Spanish.

The service is set to launch in the first quarter of 2010, Gundotra said.

Google already offers the ability to search by voice--notably with applications for the iPhone and Android phones that today work in English and Mandarin Chinese.

Gundotra said Japanese now has joined the other options for the applications, and that more will come. "In 2010, you will see us dramatically expand our efforts and support more languages," he said.

Language is key to Google's mission and operations, and the company touted its progress in the area. Mayer said Google now can translate words from any of 51 languages into any other. In 2008, Chief Executive Eric Schmidt said the company expects to increase that to 100 languages.

"We are working to break down the language barrier," Mayer said. "That focus is what unlocks the Web."

Google now intends to deliver customized search results even to those searching its site without having signed into a Google account.

Google keeps a history of your Web searches for up to 180 days, using what it says is an anonymous cookie in your browser to track your search queries and the results you most frequently click on. For several years it has allowed those with Google accounts to receive customized search results based on that history, but now even those without Google accounts will receive tailored results based on a history of their search activity, Google said in a blog post late Friday.

For example, Google described in a video how the query "SOX" might signal one type of search intent coming from baseball fans in Boston or Chicago, and another type of intent from an accountant closing the books on the quarter. Based on that particular person's search profile, Google can promote links to baseball scores or Sarbanes-Oxley details higher in search results than other links affiliated with those queries.

This, of course, is not just about search results. By building a profile of past searches, Google can also gain insights into what kinds of advertising you're most likely to favor, therefore placing more targeted (and expensive) ads alongside those search results

Privacy advocates will likely be put off by the fact that this is an opt-out rather than opt-in service. Beforehand, the customized search results were only available to those who were signed into a Google account, and although Google has always stored the search history of anyone who visits its site, it didn't change individual search results based on that history.

Google was careful to describe the procedure for opting out of personalized results, and emphasized that it doesn't know who specifically is attached to a given set of search queries. But in essence, even those who search Google without being signed in can now be used to help Google improve the targeting of its search results and its ads.

An overview of how Google arrives at Personalized Search results.

(Credit: Google)

To show support for the global fight against AIDS, both Google and Twitter changed up their sites a bit Tuesday.

If you go to Google.com, you'll find a link under the search box that leads to several resources where you can learn more about AIDS, volunteer to fight the disease, and donate money to fight AIDS. It's no small contribution to the cause--Google's home page is undoubtedly driving considerable traffic to all the organizations the company lists.

Twitter has turned red for World AIDS Day.

(Credit: Screenshot by Don Reisinger/CNET)

Twitter has introduced a more obvious change to its site. Whenever a user adds the hashtag #red to their tweets, the message they update their status page with will be displayed in red to followers. Users can also add the hashtag #laceupsavelives to turn their tweets red. The change is part of the Turn Red initiative, which aims at battling AIDS in Africa.

You can learn more about Join Red and the fight against AIDS on the organization's Twitter page.

Google's new search interface is being tested among small groups of users.

(Credit: Screenshot by Tom Krazit/CNET)

Google's new search results page brings more search options to the fore amid brighter colors that nod at recent changes made by its rivals.

Small groups of Google searchers over the past week or so have seen the new design as the company tests the new user interface, but not everybody was able to gain access to the interface. On Wednesday, Gizmodo published some tips on how to force Google into serving the new pages, and therefore we can bring you some screen shots of the new look and feel for Google search.

The first thing you'll probably notice is the left-hand rail, which has Google's search options feature presented in full color and in permanent position: previously, you had to toggle the search options feature at the top of the search results page, and the links were presented in Google's classic spartan blue. The search bar at the top of the page also has a big blue "Search" button in place of the gray button that used to occupy that space.

Marissa Mayer, vice president of search products and user experience at Google, told Search Engine Land last week that the new pages are designed to eliminate inconsistencies in how Google presented search results after it added several new elements to the page. But it also mimics what Yahoo and Microsoft have been doing with their search results pages, focusing on presentation and new ways to sort results.

Let us know what you think of the new search user interface. Fair warning: the procedure described by Gizmodo caused a few temporary issues for some CNET folks that cleared up upon a few refreshes. Google will be testing the new pages for several weeks before deciding what will make the final cut for all searchers.

We all know the mathematical adage detailing how many words a picture is worth. The principal applies equally to search results. Search for anything in Google, Bing, and Yahoo and see how long it takes your eyes and brain to max out on all the written input. (The concept of text fatigue also applies to blog posts, which is why we've included a nice, large picture near the top of ours.)

(Credit: Screenshot by Jessica Dolcourt/CNET)

If you browse the Web with Firefox (Windows | Mac), the free extension SearchPreview (Windows | Mac) breaks up text blocks by inserting thumbnail images of the site's homepage to the left of the text, where your eye naturally goes. If you're unsure which of the many returned links you really want, a glance at the thumbnails could settle the dispute.

If the thumbnail image scenario sounds familiar, that's because until recently SearchPreview has been known as GooglePreview. An upgrade to version 4.0 added support for Microsoft's Bing search engine and repositioned the product with a name change.

As you use SearchPreview, you may notice some results tagged as SearchPreview's sponsored links. These are the developer's way of recouping costs, but you can disable the sponsored results from the Options menu of the add-ons manager.

(Credit: Google)

Google released a new version of the free Google Mobile App for Symbian Series 60 (S60) phones on Monday. The update gives phone owners the ability to press the Talk button and speak search terms into the phone. While a new feature to the S60 operating system, users on other platforms, like BlackBerry and iPhone, have been able to turn speech into search results for some time.

The new Google Mobile App shows up as a shortcut widget on the Nokia home screen, which makes sounding out searches on those Nokia N and E series handsets faster than on other mobile platforms, where you must open the Google Mobile App to begin a search.

Whether you talk or type, Google Mobile App uses GPS or cell tower triangulation to fill in your location and find the closest whatever-it-is nearby. This is consistent with Google Mobile App for other platforms, though Windows Mobile is the only other one that also uses the home screen plug-in.

In addition to adding digital ears to search, Google has made them more global. Mandarin Chinese has joined Google's speech recognition database, so Nokia seekers can speak queries in English or in Mandarin. Google warns that the Mobile App is better at distinguishing certain accents better than others; a Beijing lilt may search more successfully than southern-flavored speech, for instance.

Mandarin recognition is currently only available for Nokia phones, but Google says in an official blog post that they're working to expand the capability to other mobile platforms, like Google Android and iPhone. Also, not every S60 owner can take advantage of the new Google Mobile App, only those running version 3. The app is not yet supported on touch screen phones, which run version 5 of the system software.

You can download Google Mobile App for Nokia S60 by pointing the mobile browser to http://m.google.com.

Google on Wednesday announced that its search results now feature an option allowing users to view formatted PDFs from within their browser.

Although Google's search results have long featured a "View as HTML" option for documents using the Portable Document Format standard, the company in a blog post said that "option loses some of the formatting from the original PDF, such as graphics, tables, fonts, and other elements."

To solve the issue, a new "Quick View" option has been added to some PDFs in search results. When a user clicks on the link, the full PDF file is displayed in the browser with all its formatting intact. The viewer is based on the same service built into Gmail and Google Docs.

Google's Quick View in operation.

(Credit: Google)

According to Google, it has been adding the Quick View feature to results since July. Currently, more than 50 percent of the PDFs in Google's index display that viewing option.

Google also said it plans to use the viewer for "more documents and file types."

The new options on the left side of a regular Google search results page emphasize how important presentation has become in search results.

(Credit: Google)

Google has added a few new filters to the search options panel it introduced last May, emphasizing speed and continuity on its search results pages.

The "show options" link at the top of a Google search results page brings up a number of filters on the left side of the search results page that allow searchers to refine their queries, allowing them to search just for content types like videos or search results from a certain timeline. Google is gradually rolling out some new options in that panel, allowing searchers to find results from the last hour or results posted in Google Books or Google News, said Nundu Janakiram, product manager in search. Searchers will also be able to select if they want to see more results from stores or less results, depending on whether you're shopping or looking for other kinds of information.

You've obviously been able to search within Google News or Google Books up until this point, but Google thinks the new search options are useful because they won't require the searcher to leave the main search results page. Clicking on the "News" filter will present Google News result, but the look and feel of the page won't change with the new filters, Janakiram said.

This could also allow Google to sell different types of ads on the same search results page. The filters chosen by the searcher can be "a potentially helpful signal about user intent, so it does change the way ads appear," Janakiram said. It's not clear whether Google will actually use different sets of ads for different filters, however; it could just do what it currently does in removing ads from search filters where they don't make sense, such as the Timeline view.

Those features will be enabled for any Google visitor, but if you're a Google account holder and have chosen to enable Web History on your account, Google will also surface a filter that lets you refine results by pages you have already seen. This could let you more easily find a page you've already visited or make sure you exclude the ones you've already decided don't work, Janakiram said.

The new options emphasize how competition in the search business at the moment is focused on improving the presentation of search results, as opposed to better ways of indexing and ranking the results themselves. Work obviously still goes on at that level--Google is currently in the process of testing its massive Caffeine update--but much of the innovation we've seen in recent months involves the presentation of search results through graphics and a focus on the so-called "real-time" Web.

Much of Bing's early success can be attributed to its eye-catching presentation, and Wolfram Alpha is attempting to carve out a niche in search through a completely different method of presenting results. Accuracy and relevancy are arguably just as important to searchers as they ever have been, but just about every search provider has articulated a desire to move past the "ten blue links."

The new features build on a couple of other enhancements Google has trotted out in recent days, adding a "Hot Trends" link to the Google Trends page for emphasizing popular searches as well as "jump to" links that direct a searcher to a Web page beyond the home page for that topic.

Google's greatest challenge as it heads into its second decade may very well be innovating without ticking off Uncle Sam.

The position taken by the Department of Justice two weeks ago on the Google Books search settlement marked the second time in about a year the U.S. government has taken an active step to rein in one of the tech industry's signature companies. Google now is in the process of renegotiating a deal it once called "a historic settlement," one that gave it sweeping and exclusive rights to digitize certain kinds of books that competitors and activists feared would allow Google to gain more control of access to digital information.

Unlike the famous technology antitrust cases of the last century--IBM, AT&T, Intel, and Microsoft--Google has not drawn government scrutiny based on allegations of past conduct. Rather, the focus is on what the company might do or is about to do, which should be both comforting and troubling for Google executives.

Such scrutiny means that almost every product launch will be examined for signs that Google is abusing its dominant search position by moving to control other areas of the tech world. You can double that level of scrutiny for any potential acquisition. And it also means the company will need to aggressively court government officials, who are getting an earful from companies like Microsoft, Amazon, and AT&T about what a bully Google has become.

In short, it means Google has entered a new era where its cost of doing business has changed.

"I don't think Google wants to start imposing limits on itself," said Eric Clemons, a professor at the University of Pennsylvania's Wharton School of Business. But it may not have any choice, if the Justice Department proves continually willing to impose such limits on Google's behalf.

Scrutinizing what it is about to or might do
How did Google get here, without actually being accused of anything more than hubris? Just 10 years after its founding, and five years after its IPO, Google has evolved from scrappy start-up to innovative game-changer to domineering titan in less time than it took to prosecute the Microsoft antitrust cases of the 1990s.

There's no real hard-and-fast rule concerning when a company turns the page from big to too big. That's because it's not illegal to be big: what is illegal is using that heft to expand into other areas or fend off challenges to a company's dominance.

Microsoft, for example, was prosecuted under two separate areas of the Sherman Act, the 100-year-old law that governs antitrust investigations in the U.S., said David Turetsky, co-chair of the antitrust practice at Washington, D.C., law firm Dewey & LeBoeuf and a former deputy assistant attorney general for the Justice Department during its prosecution of Microsoft. It was investigated for the onerous contract terms it imposed on its customers as well as its plans to bundle Internet Explorer with Windows, which conflict with Section 2 of the Sherman Act.

No allegations like that have been made against Google. Instead, Google has been scrutinized for what it is about to do or could do, such as when it proposed the purchase of DoubleClick in 2007 (which was approved) or its proposal of a search advertising deal with Yahoo (which came within hours of a Justice Department lawsuit before Google folded its hand).

In the Yahoo case, the government objected to a deal that would have put almost 90 percent of the search advertising market in the hands of a single company. It had several bones to pick with the Google Books deal, which it called "anticompetitive" and an abuse of the class-action settlement process.

Now that it has had its hand slapped twice, Google's lawyers will have to advise caution among product development executives, adding antitrust concerns to the normal practice of getting legal to sign off on a new product or strategy. Kent Walker, Google's general counsel, in a fine example of trying to make lemonade from lemons, argues that might actually burnish Google's image before the government.

Kent Walker, general counsel for Google.

(Credit: Google)

"The whole shift of antitrust law in the U.S. and Europe of the last 20 years has been in the direction of protecting consumer welfare," Walker said. "That makes it relatively easy to advise management, because the goal is consistent with the values that (Google co-founders) Sergey (Brin) and Larry (Page) set out, which is about making the Internet an open place."

Walker admitted Google had a few lessons to learn following the Justice Department's indication that it would sue to prevent Google and Yahoo from pooling their search advertising. Google has embarked on an educational campaign since then, posting videos featuring chief economist Hal Varian explaining how Google's ad auction process works and devoting more time to projects like the Data Liberation Front, which works to make sure Google users have simple ways to export their data from Google should they wish to move on. This has probably helped Google in the court of public opinion, but it's not clear whether government regulators have been swayed.

One concern often advanced by competitors and detractors is that Google can use the cash cow it enjoys from its dominant position in the search advertising market to subsidize businesses in ways that other companies can't afford to do, thereby driving everyone else out of the market. Last quarter, Google generated $1.4 billion in cash. The Android operating system and Google Apps often come up in those types of discussions, because there aren't many companies who could afford to invest that much money in software development only to give the product away for free.

Walker, as might be expected, doesn't buy that line of thinking. "The Internet is not a place where you can lock out competition," he said, arguing that the barriers to entry in the Internet applications market are low and that Google could never get away with trying to charge money for things it previously gave away for free, government scrutiny or not.

Of course, Google didn't exactly invent the concept of the loss leader. Microsoft does something similar with its Internet strategy, using the immense profits from sales of Windows and Office to allow it to lose money competing with Google in search and Yahoo on portal services.

Google is taking this increased government scrutiny seriously. It has built a D.C. operation from scratch over the past few years to help lobby its case in Washington, and lawyers assigned to product teams now consider antitrust issues along with the usual assortment of legal concerns.

And Google CEO Eric Schmidt has taken a very active public policy role over the last few years, stumping for President Obama during his campaign in 2008 and serving as a technology adviser to the administration. It's not clear whether that has had any effect--the current administration seems much more intent on scrutinizing big business than the Bush administration--but a fair number of ex-Googlers have taken positions in the administration. Perhaps most notable is Google's former head of global public policy and government affairs, Andrew McLaughlin, who is now deputy chief technology officer for Internet policy under federal CTO Aneesh Chopra.

The case against Google
Wharton's Clemons think Google's antitrust issues start with its bread-and-butter business. Google, he believes, could be taken to task if the Justice Department thinks about Google's search business as a distribution channel, rather than advertising-based.

Clemons' argument is that Google controls so much of the information that is presented to Internet users through both search results and search advertising that it creates a disincentive for companies to do anything that runs afoul of Google. As of August, according to ComScore, Google held 64.6 percent of the U.S. search market, compared to Yahoo's 19.3 percent and Microsoft's 9.3 percent.

It also means, according to Clemons, that Google's practice of allowing companies to purchase ad keywords bearing their competitors' trademarks forces companies to overpay for ad keywords they arguably own and can deceive the Web searcher if a competitor's ad is placed among the prominent search ads accompanying a search for the original company's trademark. Lawsuits over this practice are pending, and could potentially add another headache for Google's lawyers down the road.

"Monetization of misdirection frequently takes the form of charging companies for keywords and threatening to divert their customers to a competitor if they fail to pay adequately for keywords that the customer is likely to use in searches for the companies' products; that is, misdirection works best when it is threatened rather than actually imposed, and when companies actually do pay the fees demanded for their keywords," Clemons wrote as a guest author on TechCrunch in March.

Not surprisingly, it's a controversial and perhaps extreme view; Clemons was taken to task by several search industry veterans including Search Engine Land's Danny Sullivan regarding two articles he guest wrote for TechCrunch espousing that view. "Search advertising is one of the most powerful forms of advertising precisely because it does not misdirect searchers, nor interrupt them but instead provides answers that they seek," Sullivan wrote in response to one of Clemons' articles.

There's no indication the Justice Department is looking seriously at this argument, but it's an example of the kind of opposition Google can expect to face if it remains the dominant player in the search market.

In Google we trust?
Google is an unlikely robber baron. The company's public image is carefully crafted to seem warm, open, friendly, and respectful. No doubt, Google execs have learned from their competitor's mistakes: Microsoft continues to be haunted by what some saw as the company's arrogant and condescending performance during its own antitrust defense.

"They (Google) seem to be able to move forward with energy and creativity despite whatever antitrust issues they've encountered," Turetsky said.

However, Jeff Jarvis, author of the book "What Would Google Do?", believes there is a disconnect between how Google sees itself and how a growing portion of the public sees Google. "Google thinks (it's) Snuffleupagus--big but cuddly and good--and just doesn't realize that some people see it as a potential bully and so it has to act accordingly. With size comes responsibility," he wrote in a blog post last week.

Few deny Google's important contributions to advancing the story of the Internet. Even Google's fiercest critics of the book search settlement agree that a digital library would be a tremendous asset to the world, and that few other companies, organizations or governments seem willing to invest the time, effort, and money into such a product.

But Google, simply because of its success in becoming synonymous with "discover information on the Internet," makes people uneasy about granting it a further lock on information. And with the number of companies with a serious chance of challenging it in the search market set to dwindle to one, that uneasiness is not going away.