Update 2 p.m. PDT: I added more detail and examples of searches that stumped Google.
SAN FRANCISCO--Udi Manber sums up Google's core challenge with this description of people's expectations: "Here's what I say, now give me what I need."
In other words, the company must use computers to comprehend humans, said Manber, the vice president of engineering in charge of Google search, in a speech at the Gilbane Conference here Wednesday.
"Ideally, we would understand your question, we would understand all knowledge, and match the two," Manber said.
Udi Manber, head of search engineering at Google, speaks Wednesday at the Gilbane Conference.
(Credit: Stephen Shankland/CNET News.com)That's not possible today, though, so Google takes a shortcut: Google tries to analyze and summarize all content, extend a user's query into a summary version, and then match the two.
That sounds like a pretty long shortcut, but clearly Google has set its standards and goals very high. "We strive to answer every question, in every language, in a personalized fashion, in less than 100 milliseconds, for free," Manber said.
In Manber's view, humans are a puzzle only beginning to be unlocked. "The 20th century was about conquering nature. The 21st will be about understanding people," he said, and computing is following suit. "The largest computing clusters in operation today are doing search, e-mail, social networking."
Google starts opening up
Google is notoriously secretive about exactly how it decides which results to show in response to a particular query--a subject of high interest to companies counting on high placement or people hoping embarrassing Web pages will fade away--but the company has begun opening up. Manber promised in a blog posting in May to shed more light on search quality in coming months.
Manber shared several details about Google's search quality process in his speech. For one thing, he said, there are more than 100 "signals" the company uses to determine the order of search results. Signals can be anything from language to location to a person's previous search behavior--the latter only if the user enabled Google's search history feature that personalizes results.
He also said the company has a team of "dozens" who do nothing but analyze the quality of search results, where quality is measured by hundreds of charts. These employees support the engineers who try to improve the search results, and Google wants those engineers to experiment with new search quality methods, Manber said.
Frictionless engineering
"The basic idea is to remove friction from engineers...An engineer with an idea does not ask for permission," he said. Instead, the engineer tries the experiment, and Google meets once or twice a week to judge by the data whether the changes should be incorporated into Google's main search results.
These experiments take place on a dedicated cluster of servers, Manber said.
"My group at Google has at its disposal many thousands of machines, with storage measured in petabytes," Manber said. "This is just for our own use, not for satisfying your queries."
Google also tests search algorithm changes on users, different groups of whom receive different search results through a comparison process called split A/B testing.
The end result: Google adopts search changes quickly and frequently. Google made 450 search algorithm changes in 2007, for example.
"We opened the way for any engineer to go improve things. Mostly because it's based on data," Manber said. "There is no separation of research and development. Everyone does both."
Tough nuts to crack
Manber appears to take a perverse pleasure in difficult searches, relishing the fact that expectations for search match the rising capability and size of Google's infrastructure.
He cited as examples out a series of searches whose intent generally seemed clear enough to a human: southeast utah news-airplane crash 10/25/06, hairstyles for ears that stick out, inflammation and pain under my rib, what is answer to this math problem 6x/10x, how many calories in a pound, if real number show else error blank excel.
Of that collection, Google only provided good answers to the inflamed rib query, he said.
Straightforward queries also can be tricky. Google uses context to gauge what exactly "GM" stands for General Motors in the query "GM cars" but genetically modified in the query "GM foods."
Google offers various advanced search options, but its general policy is to use its single search box for everything.
"We have to understand as much as we can user intent and give them the answer they need," Manber said.
Q&A Search has become central to the functioning of the Internet, but Udi Manber isn't the kind of person who takes that for granted.
"I don't have to tell anybody around here that search is important. That's a very nice luxury to have," said Manber, the Google vice president in charge of search quality.
Udi Manber, Google VP, engineering
(Credit: Google)Search quality may seem like an unassuming element of Google's operations, but in fact it's at the core. Manber oversees the company's search algorithm--all the different inputs Google weighs to judge which Web sites to rank highest in search results.
Manber's work has been highly secret, partly because search is central to Google's competitive advantage and partly because Google doesn't want people gaming the system to get artificially prominent results. But the company has begun sharing a smidgen, including an opening blog post by Manber in May. I talked to him at Google headquarters recently.
How mature is search today on the Internet? Are we 5 percent of the way done with the problem? Ninety percent?
My best analogy is that a 15-year-old thinks he's very mature. A 19-year-old thinks he's extremely mature. Every few years you learn that you were not mature before. Search on the Web is about 15 years old, and obviously we were much more mature than we were 5 years ago and 10 years ago and 15 years ago. One way to put it is that it's science fiction every 5 years. What's possible today to me was science fiction 5, or definitely 10 years ago. What was (ordinary) 10 years ago was science fiction 15 years ago. The development is really pretty amazing. It surprised even me. I expect a certain level of progress, and we're actually surpassing it.
You were at the University of Arizona, then Yahoo and Amazon, then A9, then you moved to Google in 2006. Is there anything you've learned from looking at it from different perspectives, or have you been just tackling the same thing with different phone numbers on your business card?
It's the same problem, and I've looked at it from many different angles. It's bigger here, and it's better here. We have a team that's beyond any other team I've ever been with. We put more resources into it. I don't have to tell anybody around here that search is important, and that's a very nice luxury to have.
I remember the old days of AltaVista and HotBot and WebCrawler some of these other search engines--days when search was really very primitive.
I remember starting those things. They looked very sophisticated and mature at the time, which is my point about the 15-year-old.
It's clearly become a lot more usable. But even 10 years ago, everybody hadn't been trained to think the way we get information is we go to a search box and type something in. Now that seems abundantly obvious. What 10 years from now is going to look stunningly obvious as having a search box is today?
It was clear to some people. I don't want to brag too much, but it was clear to me. That's why I moved to search in the early 1990s, because everybody was talking about the information revolution. It was very clear that to have an information revolution, it's not enough to store the information and move it around, you have to find it. I know a lot of people at the time who were talking in those terms--that's going to be the revolution. The ability to find things among huge amounts of information is the key factor. So while nowadays it's completely obvious, even 6 or 7 years ago it was not obvious. I think the reason Google is so successful now is because it was obvious to (co-founders) Larry (Page) and Sergey (Brin) 10 years ago, they put in all the effort, and they're still doing it now.
Don't take that for granted. It was not that well understood, but it was understood by some people. When I started working on search when I was in academia and I said I'm working on search, they looked at me and said, "What do you mean you're working on search? Did you lose something?" In the early 1990s, even, very few people worked on search, because search was done by professionals in various limited domains. There was legal search, there was medical search, there was chemical search, and some limited news search. And it was done by a searcher--professional people. You tell them, "This is what I want to find," and they find it for you. I went to trade conferences with searchers. The idea that people will do the search themselves--that it'll democratize the whole thing and you don't have to go to a professional--that's the revolution.
I think that'll advance much more because you'll do more searches. There are a lot of things you don't search for now, because you don't expect Google will know or that the search engine will find out. We are finding that user expectations grow. The kind of searches people do now are more complicated than the kinds they were doing five years ago. People expect a lot more from us.
Ten years ago, if you actually found an answer to some specific question, it was, "Hey, look at this, it's so cool!" It was an event. Nowadays if you don't find exactly what you want in the first or second result, something is wrong. That's nice. The expectation is that we'll do it.
... Read more
Google has concluded it's been a little too secretive about the inner workings of its search engine.
The company has deliberately stayed mum about the algorithm that decides what to put at the top of the search results list, in part because the company doesn't want competitors copying it and in part because it doesn't want Web sites gaming the system, said Udi Manber, the vice president of engineering in charge of search quality, in a blog post Wednesday. Now, though, it plans to share a little more.
Udi Manber, Google VP, engineering
(Credit: Google)"Being completely secretive isn't ideal, and this blog post is part of a renewed effort to open up a bit more than we have in the past," Manber said. "We will try to periodically tell you about new things, explain old things, give advice, spread news, and engage in conversations."
The blog post mostly just outlines the search quality effort, but we'll have to wait for future blog posts for the real dirt. But Manber gives a glimpse of some of the factors--internally called inputs--that Google weighs.
"The most famous part of our ranking algorithm is PageRank, an algorithm developed by Larry Page and Sergey Brin, who founded Google. PageRank is still in use today, but it is now a part of a much larger system. Other parts include language models (the ability to handle phrases, synonyms, diacritics, spelling mistakes, and so on), query models (it's not just the language, it's how people use it today), time models (some queries are best answered with a 30-minutes old page, and some are better answered with a page that stood the test of time), and personalized models (not all people want the same thing)."
In 2006, Google hired Manber from Amazon.com, where he led the company's A9 search engine work. Before that he worked at Yahoo.
Manber also said humans and automated tools constantly evaluate how well Google is doing, and it constantly rolls changes into the search algorithm--Google made 450 changes to its algorithm in 2007, he said. Some were minor, he said, such as correctly understanding acronyms in Hebrew, and some were major, such as a big change to PageRank in January, he said.
The company also has begun opening up a bit to the press. It shed some light on search challenges during its Google Factory Tour event Monday, and I plan to publish a Q&A with Manber shortly.
- prev
- 1
- next
