Researchers at Microsoft have developed a tool to scrub search engines of major Web sites that pollute search results and ultimately help clean the Web of spam.
Called Strider Search Defender, the tool is designed to dig out Web pages that are a front for spam Web sites, according to a paper published by Microsoft researchers on Thursday. These Web pages typically reside on blogging sites and other services that provide free Web space, the researchers said.
Spammers soil the Web with countless links to their spam fronts in order to gain a higher ranking in search engines. "By cleaning up Web search, hopefully we can discourage spammers from cluttering the Web with spam," Yi-Min Wang, principal researcher at Microsoft, said in an interview.
Microsoft's tool doesn't find spam the traditional way, by looking at the site's content. Instead, it turns the spammers' activities against them by using search engines to find links to potential spam pages. These links are often posted as comments on blogs, in online discussion forums and in guestbooks, also called "comment spam."
Search Defender starts with a list of confirmed spam Web addresses. A "Spam Hunter" part of the tool runs those addresses through search engines to find pages that link to the spam sites, using the "link:" query tag. Additional spam URLs found on those sites are, in turn, run through the Spam Hunter, resulting in a long list of potential spam sites.
Then, using another Microsoft research project, Strider URL Tracer, false positives are filtered out and a list of Web pages that redirect to spam sites is compiled. Strider URL Tracer actually visits each one of the Web addresses found by the Spam Hunter to see if it redirects to a secondary spam page.
"We use search engines to find them," Wang said. "Spammers are basically telling us: Here are my spam URLs."
Spammers use various online services to host spam fronts, including free Web hosting providers such as Tripod, Angelfire and Yahoo's Geocities, Microsoft said. Blogging services are also often abused, Google's Blogger at blogspot.com is especially popular, according to the research report.
"Our preliminary investigation shows that spam blogs hosted on blogspot.com appear to be particularly widely spammed and effective against search engines," the report said.
Microsoft's researchers are working with the MSN Search team to see how the search service could be cleaned up, Wang said. Additionally, he called on the Web community, especially the operators of blog and free hosting sites, to cooperate to combat the Web spam problem.
"In the end it is all about protecting the search engines. Because if the spam doesn't show up in any search engine result the spammer will not receive traffic," Wang said.
The report that you said was released by MS today (MSR-TR-2006-97) is not new, just updated. I was first placed on the web several months ago, early April. It was discussed and link to on several blogs like this one. See the SEE ALSO link. <a class="jive-link-external" href="http://www.resourceshelf.com/2006/04/11/new-from-microsoft-research-strider-url-tracer-with-typo-patrol/" target="_newWindow">http://www.resourceshelf.com/2006/04/11/new-from-microsoft-research-strider-url-tracer-with-typo-patrol/</a>
EXCELENT> when I see 34,435,678 results I get discouraged
I sometimes want to see results of searches in other engines so I am also concerned if they might be deleting something I want to see. (like the Google lawsuit)
but this sounds like they will clean up the garbage. Like if I search for dogs a site might say "dogs, poodles, dashunds, collies... with ads that say: if you want to see my ***** pay $29.95 per month"
a very short synopsis of the crass things I've seen anything that will clean it up will be welcome but what about the false positives? what about those sites that accidentlly get in and don't get filtered out? I think it is a very possible situation for a lawsuit that should not be thrown out.
Web spammers SPAM for 1 purpose: to make advertising revenue. The majority of that is via Google Adsense. TagTooga.com collects adsense IDs and provides the reverse lookup both interactively and programmatically so you can see all the sites showing a particular Adsense ID. Of course, once TagTooga bans the spammer from it's site, that information is no longer available -- TagTooga.com uses the reverse lookup to automatically delete all submissions/domains/URLs with the Google ID, and to prevent future submissions using banned IDs. (see <a class="jive-link-external" href="http://geekswithblogs.net/chilkat/archive/2006/03/16/72540.aspx" target="_newWindow">http://geekswithblogs.net/chilkat/archive/2006/03/16/72540.aspx</a> and <a class="jive-link-external" href="http://www.tagtooga.com/pg/DownloadSites" target="_newWindow">http://www.tagtooga.com/pg/DownloadSites</a> for more information...)
EnerG2 opens a plant to make an engineered carbon that will improve performance of energy storage devices and make storage for start-stop hybrid cars less expensive.
Tommy Jordan, the man who shot his daughter's laptop for YouTube, gets a visit from police and child protection services. Oh, and Good Morning America.
As UC Berkeley students, the co-founders of "Back to the Roots" discovered they could grow mushrooms using recycled coffee grounds. Now their mushroom kit sells at grocery stores across the country.
For people who don't have time to tend a Zen garden, the Zen Table will handle the work for you. The table is filled with silicone beads and a robotic system that "rakes" images into the sand.
The Washington State Senate passed a bill that would charge electric car owners $100 per year to compensate for not paying gas taxes. The bill still has to pass the House.
(MSR-TR-2006-97) is not new, just updated. I was first placed on the web several months ago, early April. It was discussed and link to on several blogs like this one. See the SEE ALSO link.
<a class="jive-link-external" href="http://www.resourceshelf.com/2006/04/11/new-from-microsoft-research-strider-url-tracer-with-typo-patrol/" target="_newWindow">http://www.resourceshelf.com/2006/04/11/new-from-microsoft-research-strider-url-tracer-with-typo-patrol/</a>
but this sounds like they will clean up the garbage.
Like if I search for dogs a site might say "dogs, poodles, dashunds, collies... with ads that say: if you want to see my ***** pay $29.95 per month"
a very short synopsis of the crass things I've seen anything that will clean it up will be welcome but what about the false positives? what about those sites that accidentlly get in and don't get filtered out? I think it is a very possible situation for a lawsuit that should not be thrown out.
BMR777
<a class="jive-link-external" href="http://www.webringamerica.com" target="_newWindow">http://www.webringamerica.com</a>