Google sheds light, dimly, on search quality

Google has concluded it's been a little too secretive about the inner workings of its search engine.

The company has deliberately stayed mum about the algorithm that decides what to put at the top of the search results list, in part because the company doesn't want competitors copying it and in part because it doesn't want Web sites gaming the system, said Udi Manber, the vice president of engineering in charge of search quality, in a blog post Wednesday. Now, though, it plans to share a little more.

Udi Manber, Google VP, engineering

(Credit: Google)

"Being completely secretive isn't ideal, and this blog post is part of a renewed effort to open up a bit more than we have in the past," Manber said. "We will try to periodically tell you about new things, explain old things, give advice, spread news, and engage in conversations."

The blog post mostly just outlines the search quality effort, but we'll have to wait for future blog posts for the real dirt. But Manber gives a glimpse of some of the factors--internally called inputs--that Google weighs.

"The most famous part of our ranking algorithm is PageRank, an algorithm developed by Larry Page and Sergey Brin, who founded Google. PageRank is still in use today, but it is now a part of a much larger system. Other parts include language models (the ability to handle phrases, synonyms, diacritics, spelling mistakes, and so on), query models (it's not just the language, it's how people use it today), time models (some queries are best answered with a 30-minutes old page, and some are better answered with a page that stood the test of time), and personalized models (not all people want the same thing)."

In 2006, Google hired Manber from Amazon.com, where he led the company's A9 search engine work. Before that he worked at Yahoo.

Manber also said humans and automated tools constantly evaluate how well Google is doing, and it constantly rolls changes into the search algorithm--Google made 450 changes to its algorithm in 2007, he said. Some were minor, he said, such as correctly understanding acronyms in Hebrew, and some were major, such as a big change to PageRank in January, he said.

The company also has begun opening up a bit to the press. It shed some light on search challenges during its Google Factory Tour event Monday, and I plan to publish a Q&A with Manber shortly.

[sethv]