Comments on: Crawl the Web with Wales' open-source search tool

Wikipedia founder Jimmy Wales announces the release of an open-source Web crawling site called Grub.

[ben::zen]

Grub???

What about the GRand Unified Bootloader
(GRUB)??? Why two grubs? This makes no sense. At
least it isn't weevil, or we'd all be making
Master and Commander jokes right about now...

[ir8piir8]

That's neat, but your sites broke...

I believe that this is the best way to crawl the web, but fix your website! There is no contact link and I got MySQL errors when registering my team. So I'll leave my comment publicly. Feel free to reply.

[flowgar]

faroo.com goes even a step further

www.faroo.com goes even a step further: distributed crawling and distributed search.
A distributed crawler makes sense only, when the index is distributed too.