Comments on: How to evade Google search

Companies must take steps to ensure that the search giant's prolific Web bots don't grab content they want to keep hidden.

[gggg sssss]

Nice for Dell

They got the cache cleared in a day. Took me a month. Now I know my place in the universe.

[assman]

in order to get it done fast..

you need to call.. or visit google's headquarters, which dell undoubtedly did

[whogrant]

Google's own tips don't even work

My site got hit by being #1 for the "Iraq flag" Google image search. So I followed Google's webmaster tips for preventing images from being indexed by using a robots.txt file. It was supposed to happen the next time they indexed my site (which happens very frequently for my site - every few days). Instead my site remained #1 for three months afterwards.

While this is not the same as trying to not get indexed in the first place it leads me to think that I probably wouldn't trust all their tips to work as advertised. Who know's maybe they index and cache everyone's content anyway regardless of the "don't index me" hints. Then once in a while stuff accidentally gets into the index, or doesn't get removed quickly. Plus if your stuff is in their cache who knows how long it will stay there for the Department of Justice or anyone else to subpoena them for?

I once had someone contact me about my resume which was on my web server but I new I had never linked to from my site our shared the link with. Clearly the person found the file just by guessing the URL. I wouldn't be at all surprised if Google and others do this kind of thing to uncover hidden content.

For everyone, the working assumption should be, if you don't password protect access to your content then it will probably end up indexed on Google or on some other search engine index one day.

[gerardocb]

dangerous?

well... the file must be robots.txt, and some people think this can be dangerous, 'cause other ppl can know in an easy way (just retrieving the robots file) where your valuable content is.

tech republic locksmith

And, don't forget about archive.org

USE robots.txt (NOT robot.txt)!!!

This article mis-states basic SEO 101 subject matter regarding
robots.txt, this site accurate info:

http://www.searchengineworld.com/robots/robots_tutorial.htm