New search engine Cuil takes aim at Google

Cuil's homepage.

There's a big new search engine launching Monday: Cuil. Developed and run by the husband-and-wife team of Stanford professor Tom Costello and former Google search architect Anna Patterson, it's pitched as bigger, faster, and better than Google's flagship search engine in pretty much every way. See video interview with Tom Costello, below.

I have not had a chance to spend much time with the engine. I'm getting open access to it the same time you are. I did get a preview. It's a very serious effort, and it has enough funding to get off the ground and become a player.

The most important difference between Cuil and Google is its ranking system. Rather than assigning priority to pages based on inbound links as Google does ("Pagerank"), Cuil analyzes the content of Web pages to divine their relevance to a search query. Costello bristled when I asked if this was a semantic search engine like PowerSet (recently sold to Microsoft). Costello said Cuil's search is "contextual," and that, "we're trying to understand the real world, not the Web."

Cuil really does a better job of displaying search results.

(Credit: Cuil)

What this means, in the real world, is that Cuil results are automatically categorized. When you search for a common name, for example, Cuil will give you a result page where results for different individuals with that name are groups under tabs. It will also break out sub-topics related to each name. In Cuil's canned demo, if you search for "Harry," there are different tabs for "Harry Potter" and "Prince Harry of Wales." On the Harry Potter tab, you'll get further sub-links devoted to actors, Gryffindor dorm-mates, etc. "We have a strong ontological commitment," Costello told me, meaning that parsing search results into readable chunks is a very big part of the Cuil value proposition.

The service also displays images from Web results whenever possible. It all adds up to search results pages that are much more attractive, and useful, than Google's.

Another potential advantage of the context-based search is that it allows Cuil searches to be more respectful of user privacy. Unlike Google, which simply has to track every single click to refine its index, Cuil's context-based search does not. In practice, the distinction may be moot because Cuil will need to track clicks to see if their results are actually working for people, but it could serve as a marketable distinction.

Context-based indexing also presents a juicier target for search spammers, but as Costello says, "that's a success problem."

It's one thing to have a nice interface and show users good results, but the size of the Web index that the engine has access to matters a lot as well. And this is where Cuil makes its boldest claim. Costello says that the engine is launching with 120 billion pages indexed, well over the 40 billion he says Google has (although see Google's latest bluster about the company's power at Web indexing). Costello also claims that Cuil's Web crawler is three times faster than Google's, although it wasn't clear to me if he meant that is per search computer or for the entire system. Compared with Google's globe-spanning data network of data centers, some literally set up near dams so they can tap hydro power more efficiently, Cuil's two puny data centers hosting less than 2,000 PCs total will have to run pretty fast to outpace Google's crawlers.

Cuil will launch on Monday, and in a refreshing (and gutsy) move, the site is just plain launching. There's no weasely "beta" tag applied to the service. Costello thinks it'll be good enough to use from day one.

It won't, though, be as complete as Google. While Google has had failures in extending its brand (Froogle, Google Base), its collection of services that are affiliated with its mainstream search product, like Google Maps, Image Search, and desktop search, can make switching away from Google difficult for users. Costello realizes that Cuil needs to layer in additional services, but as he said to me, the company has to start somewhere.

Upshot: Cuil is certainly worth trying out. If you like it, services to put it in front of your face (a browser toolbar, and widgets) are coming soon.

As a business proposition, Cuil is obviously a big bet. While search is a monetizable business, it's hard to change the behavior of a generation of Web users who think "Google" is a verb. No other search engine has come close to entering the public consciousness like this. Of course, Cuil doesn't have to trounce Google on day one. It took Google quite some time to surpass Alta Vista and Yahoo in the search wars.

See also: Yahoo, Microsoft, Ask, Hakia, etc.