Spam experts at MIT lift curtain on search

CAMBRIDGE, MASS.--If there is a solution to search engine spam, no one's going to reveal it, according to experts at the MIT Spam Conference 2007 on Friday.

Nothing is going to change while there is money to be made off things like AdSense, Google's service for ads, according to speakers and audience participants debating the issue. On the other hand, the search engine that does come up with a working solution will want to keep that trick out of its competitors' hands.

"If you can't solve search engine spam, and the other guy does, your search engine is done at this point," said Bill Yerazunis, chairman of the annual MIT Spam Conference.

The situation seems to have led to a clam-up in the spam research community.

"You've heard our lead speakers about blog spam and search engine spam and that's going to get more interesting, but here's the thing. I put out the call, but I am not getting a lot of papers on search engine spam because, independently, companies think they have a profit in the solution," Yerazunis said.

"Think of this. If you were Google and you came up with a solution to solve search engine spam, would you publish it? It's a race among the big three. If Microsoft has solved it, they are going to knock out Google. If Yahoo has it, they are going to knock Microsoft out. The stakes here are billions of dollars," he said.

Yerazunis suspects that even people in the academic realm are keeping a closed lid on solutions to search spam, because the payoff is huge. If someone in academia finds it before the private companies, he's sure they're going to sell it to the highest bidder.

Because no cure is evident in the foreseeable future, education is the key to eliminating a lot of spam because much of it is originating from legitimate brands and advertisers with bad practices, said Amanda Watlington, owner of consulting firm Searching for Profit, in a speech at the conference.

Black hats, true spammers, will not be stopped as long as there is an incentive to make money through things like Google's AdSense. But legitimate brands and advertisers should really be called gray hats, said Watlington, because a lot of them are still unintentionally creating what many in the field consider spam.

Watlington doesn't believe this is a case of feigned ignorance on the part of major brands and advertisers trying to get ahead in search marketing. She thinks it's real ignorance of people not from a technical background wanting to join what is now a major business.

"You have the advertisers who are in over their heads. They are drowning in this. They are marketers and Web designers in a tech field with no SEO (search engine optimization) training. They use old, discredited tactics that they often don't even know are wrong," said Watlington.

"Feed them info, tell them they are doing wrong, and guess what? The more they know, the less likely they are to do stupid things," said Watlington.

No company wants to be BMW, said Watlington, referring to the bad press the company got when BMW's German site was blacklisted from Google for nefarious search ranking tactics.

But as more new tools are offered, companies are getting more confused.

In November, Microsoft, Google and Yahoo came together around site maps agreeing that they would be crawled. But Watlington said that her clients had no idea on how to utilize the indexing service properly. One client went to her wanting to know why he had an indexing problem. He had been feeding 400,000 pages.

"I had to explain to him how much it takes a site obeying a polite crawl to index those pages. Look, find your high margin and put those in your site map, OK? Hello?" she said.

[wbenton]

Easy Solution...

Make it illegal to profit from such a solution. (* GRIN *)

Take the profitable market away and the problem will resolve itself.

FWIW