Comments on: Out-Googling Google, a la Krugle

Krugle has managed to beat Google at its own game. The company is doing well, in part, because it's done a good job of figuring out how to profit from open source.

[royrusso]

Code Search

On that note.... a fascinating website I've seen evolve over the years is http://www.koders.com/ I would recommend it over any other code search engine, I've seen thus far.

[Yegor Kuznetsov]

Does Krugle rely on exact definitions?

Matt, great story on Krugle!

Am I right in my understanding that Krugle?s search solution is still based on keyword recognition (like solutions of other players ? Google, FAST or Convera) and has to rely on taxonomies?

But what if the keyword is misspelled? Or if you do not know exactly what you are looking for, just have a vague notion?

There are other solutions out there that deal with this problem imitating the work of human brain (we don?t look for keywords, we look for patterns). For example, Brainware possesses a unique, patent-protected technology that sets it apart from other data capture and enterprise search solutions providers. Its products are powered by the world's only engine that does not rely on exact definitions to rapidly sift through mountains of unstructured data.

Brainware's technology allows it to recognize and find data through inexact definitions, patterns and context, mimicking the way the human brain processes and sorts information.

Here?s a case study showing Brainware in action:
Fulbright & Jaworski: Leading Law Firm Searches And Shares Knowledge Base Smarter, More Accurately
http://www.brainware.com/brain_case_lawfirm.php

[CharlesSKnight]

The Top 10 Code Search Engines

On AltSearchEngines.com, the new blog from Read/WriteWeb that covers hundreds of alternative search engines, we recently did a post on our Top 10 Code Search Engines (which includes Krugle and Koders and eight more):

http://altsearchengines.com/2007/07/13/top-10-alternative-code-search-engines-montage/

Charles Knight, editor
www.AltSearchEnines.com

[kkrugler]

Re: Does Krugle rely on exact definitions?

Hi Yegor,

Thanks for the reference to Brainware - interesting technology.

As to your question about how we search, there are two key things we do (at the lowest level) that differ from typical keyword search by Google, FAST, etc.

First, we do a fuzzy parse of the code and use the resulting syntax tree to automatically classify parts of the code. This lets us separate out comments from code, and regular statements from function calls, function definitions, and class definitions.

By doing so, we can provide better search results for general queries, and also let users perform more explicit searches.

Second, when we parse code, we use different tokenizers to help with extracting sub-terms. So searching on "context", for example, finds methods named getContext, set_next_context, and context-classifier.

It would be great to support natural language searches on code, so a programmer could enter something like "I'm looking for a fast implementation of the SHA1 algorithm in C, with full unit tests" and we only returning matching hits.

Unfortunately for most code, most of the time, the level of commenting is insufficient to extract this even with a human looking at the source. And programmatically figuring this out is one of several "holy grails" for code analytics tools, but we're nowhere close yet.

So in the meantime, we focus on what can be done now, using existing technologies that scale to the size we need (2.5 billion lines of code as of today).

But now I'll have to look at how we might use alternative technologies to do a better job of searching our 120K project descriptions, which are in English (or at least the kind of English that programmers use)...thanks!