Q&A Search has become central to the functioning of the Internet, but Udi Manber isn't the kind of person who takes that for granted.
"I don't have to tell anybody around here that search is important. That's a very nice luxury to have," said Manber, the Google vice president in charge of search quality.
Udi Manber, Google VP, engineering
(Credit: Google)Search quality may seem like an unassuming element of Google's operations, but in fact it's at the core. Manber oversees the company's search algorithm--all the different inputs Google weighs to judge which Web sites to rank highest in search results.
Manber's work has been highly secret, partly because search is central to Google's competitive advantage and partly because Google doesn't want people gaming the system to get artificially prominent results. But the company has begun sharing a smidgen, including an opening blog post by Manber in May. I talked to him at Google headquarters recently.
How mature is search today on the Internet? Are we 5 percent of the way done with the problem? Ninety percent?
My best analogy is that a 15-year-old thinks he's very mature. A 19-year-old thinks he's extremely mature. Every few years you learn that you were not mature before. Search on the Web is about 15 years old, and obviously we were much more mature than we were 5 years ago and 10 years ago and 15 years ago. One way to put it is that it's science fiction every 5 years. What's possible today to me was science fiction 5, or definitely 10 years ago. What was (ordinary) 10 years ago was science fiction 15 years ago. The development is really pretty amazing. It surprised even me. I expect a certain level of progress, and we're actually surpassing it.
You were at the University of Arizona, then Yahoo and Amazon, then A9, then you moved to Google in 2006. Is there anything you've learned from looking at it from different perspectives, or have you been just tackling the same thing with different phone numbers on your business card?
It's the same problem, and I've looked at it from many different angles. It's bigger here, and it's better here. We have a team that's beyond any other team I've ever been with. We put more resources into it. I don't have to tell anybody around here that search is important, and that's a very nice luxury to have.
I remember the old days of AltaVista and HotBot and WebCrawler some of these other search engines--days when search was really very primitive.
I remember starting those things. They looked very sophisticated and mature at the time, which is my point about the 15-year-old.
It's clearly become a lot more usable. But even 10 years ago, everybody hadn't been trained to think the way we get information is we go to a search box and type something in. Now that seems abundantly obvious. What 10 years from now is going to look stunningly obvious as having a search box is today?
It was clear to some people. I don't want to brag too much, but it was clear to me. That's why I moved to search in the early 1990s, because everybody was talking about the information revolution. It was very clear that to have an information revolution, it's not enough to store the information and move it around, you have to find it. I know a lot of people at the time who were talking in those terms--that's going to be the revolution. The ability to find things among huge amounts of information is the key factor. So while nowadays it's completely obvious, even 6 or 7 years ago it was not obvious. I think the reason Google is so successful now is because it was obvious to (co-founders) Larry (Page) and Sergey (Brin) 10 years ago, they put in all the effort, and they're still doing it now.
Don't take that for granted. It was not that well understood, but it was understood by some people. When I started working on search when I was in academia and I said I'm working on search, they looked at me and said, "What do you mean you're working on search? Did you lose something?" In the early 1990s, even, very few people worked on search, because search was done by professionals in various limited domains. There was legal search, there was medical search, there was chemical search, and some limited news search. And it was done by a searcher--professional people. You tell them, "This is what I want to find," and they find it for you. I went to trade conferences with searchers. The idea that people will do the search themselves--that it'll democratize the whole thing and you don't have to go to a professional--that's the revolution.
I think that'll advance much more because you'll do more searches. There are a lot of things you don't search for now, because you don't expect Google will know or that the search engine will find out. We are finding that user expectations grow. The kind of searches people do now are more complicated than the kinds they were doing five years ago. People expect a lot more from us.
Ten years ago, if you actually found an answer to some specific question, it was, "Hey, look at this, it's so cool!" It was an event. Nowadays if you don't find exactly what you want in the first or second result, something is wrong. That's nice. The expectation is that we'll do it.
... Read more
Google has concluded it's been a little too secretive about the inner workings of its search engine.
The company has deliberately stayed mum about the algorithm that decides what to put at the top of the search results list, in part because the company doesn't want competitors copying it and in part because it doesn't want Web sites gaming the system, said Udi Manber, the vice president of engineering in charge of search quality, in a blog post Wednesday. Now, though, it plans to share a little more.
Udi Manber, Google VP, engineering
(Credit: Google)"Being completely secretive isn't ideal, and this blog post is part of a renewed effort to open up a bit more than we have in the past," Manber said. "We will try to periodically tell you about new things, explain old things, give advice, spread news, and engage in conversations."
The blog post mostly just outlines the search quality effort, but we'll have to wait for future blog posts for the real dirt. But Manber gives a glimpse of some of the factors--internally called inputs--that Google weighs.
"The most famous part of our ranking algorithm is PageRank, an algorithm developed by Larry Page and Sergey Brin, who founded Google. PageRank is still in use today, but it is now a part of a much larger system. Other parts include language models (the ability to handle phrases, synonyms, diacritics, spelling mistakes, and so on), query models (it's not just the language, it's how people use it today), time models (some queries are best answered with a 30-minutes old page, and some are better answered with a page that stood the test of time), and personalized models (not all people want the same thing)."
In 2006, Google hired Manber from Amazon.com, where he led the company's A9 search engine work. Before that he worked at Yahoo.
Manber also said humans and automated tools constantly evaluate how well Google is doing, and it constantly rolls changes into the search algorithm--Google made 450 changes to its algorithm in 2007, he said. Some were minor, he said, such as correctly understanding acronyms in Hebrew, and some were major, such as a big change to PageRank in January, he said.
The company also has begun opening up a bit to the press. It shed some light on search challenges during its Google Factory Tour event Monday, and I plan to publish a Q&A with Manber shortly.
The blogosphere and Twitter have been abuzz with talk about this article by Shari Thurow, published Thursday on Search Engine Land. The article warns of supposed dangers against the SEO tactic of "PageRank sculpting." Readers are coming away feeling reticent to employ the tactic, fearing retribution from the engines in the form of penalties. The article paints PageRank sculpting as poor usability and black hat. I can't be any more adamant about this: neither is the case.
No disrespect intended to the article's author, but this article is classic FUD (Fear, Uncertainty, and Doubt). "Nobody ever lost their job by choosing IBM," the classic FUD saying goes. "Nobody ever lost their ranking by refusing to implement PageRank sculpting" is the FUD peddled in this article.
PageRank sculpting is a powerful tactic that is worthy of inclusion in any advanced SEO practitioner's toolkit. I have written and blogged and spoken in favor of the tactic. I stand by the tactic, as do many other top-notch SEOs, and most importantly, so does Matt Cutts.
Matt Cutts, the head of Google's Webspam team, has publicly condoned the use of PageRank sculpting on repeated occasions. For example, in this SEOMoz interview:
Does Google recommend the use of nofollow internally as a positive method for controlling the flow of internal link love?
A) Yes--Webmasters can feel free to use nofollow internally to help tell Googlebot which pages they want to receive link juice from other pages
(Matt's precise words were: The nofollow attribute is just a mechanism that gives Webmasters the ability to modify PageRank flow at link-level granularity. Plenty of other mechanisms would also work (e.g., a link through a page that is robot.txt'ed out), but nofollow on individual links is simpler for some folks to use. There's no stigma to using nofollow, even on your own internal links; for Google, nofollow'ed links are dropped out of our link graph; we don't even use such links for discovery. By the way, the nofollow meta tag does that same thing, but at a page level.)
Google has even used the technique on its own properties.
So my question to Ms. Thurow is: "Have you ever conducted any testing of the PageRank sculpting technique?" We at Netconcepts have, and it works.
According to our tests, there are plenty of occasions where it can be a valuable tool, if used wisely. For example, if you have an e-commerce site and the category pages contain three links to every single product page--the product name as a text link, the product image thumbnail as an image link, and the words "View Product" as a text link--you could nofollow the image and "View Product" links and funnel more PageRank through the much more contextually relevant product name-based text links.
If SEO is going to be respected as an experimental science instead of black magic, it needs to be implemented with an experimental approach and all tactics tested for effectiveness (within the bounds of what is acceptable according to the engines). With SEO, you don't just "set it and forget" using the purported "best practices" as defined by the SEO bloggers and speakers (and sure, feel free to include me in that set).
So if Ms. Thurow wrote this piece without any testing, it's just unsubstantiated opinion--and I wholeheartedly disagree with it. :)
And I'm not the only one either. Just take a look at the other prominent SEOs who have countered via blog posts (like "Why There's Nothing Wrong With Sculpting Your Pagerank" and Sphinn comments and desphinns (click on the "Who Desphunn This" tab to see the "desphinns").
In the process of reviewing a client's Flickr account with my colleague and fellow Searchlight blogger Brian Brown, we noticed that Flickr has recently added nofollow tags to links placed within its Web site. Flickr has been one of the few social-media entities to continue to offer "link juice" from links placed with user-generated content (in this case photo descriptions), making it a viable entity for improving inbound links to a given site.
While it's understandable that Flickr implemented nofollow tags for the exact same reason other social-media sites have--misuse and spamming--it nonetheless marks another step toward the end of major social-media sites passing on PageRank.
It's not all bad news with Flickr, though...at least not so far. The nofollow tags have not been implemented throughout the whole site. While links embedded in individual photo descriptions are nofollowed, so far, links in Set and Collection descriptions continue to be free of them. Will this change? Only time will tell. SEOers everywhere are certainly hoping not.
With Google's recent crackdown on Web sites the sell PageRank--which really means selling links--a new era has begun for backlink building. In Google's eyes, links coming into a given Web site from external, quality sites increases that site's PageRank, and therefore its standing in the search engine's eyes. Until recently, there were many sites that had quality in Google's eyes (in other words, they had great PageRank) and also sold links. Anyone could get a piece of that good PR for a price.
Google is now actively lowering the PageRank of sites that deal in that sort of business, especially the larger, better known ones. These sites are often directories that require a fee to get a listing, and so far they are the hardest hit. I saw this firsthand in a listing of directories I keep. Going through the top-level directories on the list, one well-known directory's PageRank had dropped from eight to three. Another had gone from six to a flat zero. Still others directories simply don't exist anymore, most likely closing up shop to cut losses and avoid pesky questions from paying customers.
So, are there still ways to buy links to improve PageRank? Well, first off, it has never been a practice that I've necessarily endorsed. At the same time, there were--until recently--certain directories that seemed trustworthy and respected by Google as represented by good PageRank scores. Now that many (dare I say most?) of those sites have lost that endorsement, it's hard to recommend those that remain--some of the lesser-known directories that have thus far been spared. Still, a good Google hunt will turn up paid-listing directories that have decent PageRank. Whether or not such entities are a sound investment or a "proper" moral choice remains to be seen.
The best bet is to stick with directories that are free. So far, if they don't charge for the listing and have good PageRank, they can generally be considered acceptable in Google's eyes. Make sure, though, that nofollow tags aren't in place. If they are, while the listing may generate a little traffic, the listing does little to help the PR of a site.
If you really want to invest money into links, another option might be buying sites instead of links. It's easy to spend several thousand dollars on a link-buying campaign, and with Web sites possessing decent PageRank selling for as little as $10,000, at some point buying such a site makes the best sense. But don't fall into the trap of turning it into a link farm! Google will notice sooner or later, and your investment will be nullified or greatly damaged. Instead, buy a site for which you might have legitimate need and use it in that regard, making links to your other sites a secondary, but useful, focus.
After what felt like a long hibernation period, Google has recently made at least a partial PageRank update of the visible or public PageRank, often referred to as Toolbar PageRank. And boy was it noticed--not so much with the update but with the PageRank drops targeted at sites that were buying and/or selling text links.
A lot has been said already by many who have a much deeper understanding and forecast this event. What I find particularly interesting is how PageRank has become a form of currency, yet who can say what precisely that value is? Of course, I guess that is true of currency in general. Ironically, this currency is probably unknown to the majority of Web users. After all, do your family and friends outside the industry know what PageRank is? Do they even care? And more importantly, does it affect which sites they frequent?
PageRank is especially challenging since it is hard to even arrive at a consensus within the industry. Does it impact rankings? If so, by how much? Is it merely a visual representation of other signals? And is Toolbar PageRank any real indicator of true PageRank? All good questions. Like many things related to SEO and rankings, we must accept that we may never have a completely, air-tight answer. It seems that one can always point to an example that appears to defy the answers, but perhaps that is exactly how Google likes it.
Google Trends results for 'pagerank' and 'page rank'
(Credit: Google Trends)If you have been buying or selling text links and seen your PageRank drop, then you should turn to your analytics program or log stats to see if any noticeable changes appear there. If you have been tracking keyword ranking, you should check that as well to see what changes, if any, appear there. And hopefully you'll share your results with the rest of us.
At least for the time being, if PageRank rating is at all important to you, I'd stay away from buying or selling text links. Maybe this will blow over, maybe not. Unless detecting paid linking can be truly automated and handled algorithmically though, it's hard to imagine a scalable way of handling this long term. But maybe Google's hope is that by targeting highly visible sites and getting the industry talking about it, many sites will fall into line on their own out of the fear of repercussions.
As a result of all this, Google has perhaps just raised the cost of PageRank-based links. Certainly, links bought will now have to appear completely natural without any hint of commercial motivation. In this way, we can expect to see the black market of PageRank selling to grow and to see the cost of PageRank-based links to reach all-new levels.
For the rest of us, this will only reinforce good old SEO practices. Develop great content that authority sites will want to link to. Participate within your online community not only as a way to develop links, but to interact with your target audience. Make social media part of your online plan. After all, what's better than top rankings but being so well known for whatever it is you want to be known that people go directly to your site to begin with?
And clearly, as we can see from Google Trends reporting, Google PageRank falling does at least create quite a buzz.
- prev
- 1
- next
