Spying an intelligent search engine

(continued from previous page)

"The next step will be to be able to recognize and find everything in the world that isn't words on a Web page, like recognizing concepts or descriptions of events, like a plot of a story where boy meets girl. If you put in words for that now you end up with a lot of other stuff," said Esther Dyson, a blogger of Release 0.9, which is published by ZDNet and owned by CNET Networks. CNET Networks is publisher of News.com, and Dyson has personally invested in Powerset.

Why is the time right to experiment with AI and search? One of the biggest hurdles to building AI into a search engine is that it can be impractical on a large scale. The computational power needed to calculate results efficiently can be enormously expensive, critics say. But the effects of Moore's law are pushing down prices for computers, CPUs and bandwidth, and the opportunities are ripe. Search is also a lucrative business. Google, after all, is worth $6 billion annually, thanks to targeted advertisements linked to its results.

Most of the search community believes advances in Web search from Google and others will now take place incrementally, by squeezing a bit more from Google's Pagerank, or by tuning relevance, or indexing hard to find files. But for the next leap to happen, executives like Pell say, a new architecture must be built.

Taking the pulse of search
Medstory, for example, is applying artificial intelligence techniques to one area of knowledge, health care, which is a fast-changing industry rife with inefficiencies in how people get up-to-date information. Tackling a specific body of knowledge is more economical and efficient, Rappaport said. But the tricks that Medstory is using to extract more knowledge from medicine for consumers and professionals could be applied to other industries, like finance or entertainment.

Alain Rappaport

"Our job is to get rid of the noise, so the amount of information is smaller than the Web," he said. "On the other hand, we have to compute things, not just ranking and linking, and that's computationally intense."

Rappaport said one of the more recent progressions in AI has been in moving from relying on humans to catalog connections between various data to programming computers to do the work, or what he calls the automation of knowledge structure. Tom Mitchell, chair of machine learning at Carnegie Mellon University, calls it machine learning for statistical language processing, or learning algorithms that allow computers to read text.

Rappaport won't disclose the secret sauce of the company's technology; however, he said, it's a 24/7 process in computing that connects valuable pieces of information together, such as linking one document that explains symptoms of a disease to another document with analysis of a therapeutic drug for that disease.

"It's continuously extracting information it can then build itself with," Rappaport said.

The techniques surface in Medstory as serendipitous bits of knowledge that would otherwise take a Web searcher hours or days to acquire.

For example, a search on the term "lower cholesterol" returns a detailed set of results that dives into topics on a subject, such as drugs, symptoms and nutrition information. Within each subtopic, there are lists the most relevant drugs, symptoms or nutritional supplements to lower cholesterol. Under "nutrition," for example, the most relevant link is olive oil. Point the cursor over olive oil and a page pops up with the health benefits of olive oil for high-cholesterol sufferers. Unless someone already knew that, the information could take a long time to find in a search engine like Google.

Medstory was founded in 2000 as software licensed to biotechnology firms and other health care companies, but the service recently launched to the public, and Rappaport said he plans to roll it out more widely soon.

Mitchell of Carnegie Mellon has predicted that computers will be able to read the Web by 2015--he's so sure he's bet a lobster dinner on it, and a few people have taken him on the gamble.

[Drewky]

AI

Artifical Intelligence is a meaningless buzz word. A rule based system would be accurate but no where near the emotional context of artifical intelligence. Inference Engine would be just as buzzy but much more accurate. I am sure others could come up with other terms. Maybe you should hold a contest to see who comes up with the best descriptive term. I submit Inference Engine.I hear AI and I instantly think Vapor.

[WesFlash]

Face recognition has great potential

Forget finding a girl that looks like my last girlfriend and will remind me of that pain. Law enforcement searches will benefit from facial recognition. From matching a missing kid to a kiddie porn image to finding a fellon that changed his/ her identity. For law enforcement it will be an new and invaluable tool, trolling the mug shots and internet in search of possible matches to all kinds of case files.

[Sea of Cortez]

AI is just code word for vapor/hoax ware.

I agree with other poster, AI is just code word for vapor/hoax ware.

If I had gotten a dollar for each time I heard of AI and then nothing of sort was ever delivered I would be a rich man.
There is no such a thing as AI (Artificial Intelligence) software and there wont be any, because we have no idea of how to emulate complex intelligent thinking in software, there is only good software engineering.

So it is not AI that is going to make a better search engine to Google & Yahoo, it is innovative new search engine ideas which are implemented based on good software engineering work that it is going to do it. And I will tell you about a search engine that is better than Google or Yahoo,
it is called Anoox, and it is better based on these points:
1- It is powered by the Knowledge of the people
2- It is operated in an Open fashion, so NO one company owns/controls it but 100's of different company's from around the world will
It is here in case:
www.anoox.com
Ah, another reason it is better, it is also not-for-profit.

[darmik]

Goolge has pushed the limit of html search

google has pushed the limit of html search. as long as pages and sites continued to be created in a way and format that does not allow for descprtive classification any real AI will be impssoble

[Rob at Collarity]

True Natural Relevance not AI

I think AI as a search descriptor is a little oxymoronic as well. The comments here demonstrate how each person has their own contextual relationship to the keyword "AI".

I do however agree that search as a science is in its infancy and that current cpu/storage efficiencies now make it possible to deliver an events-based (usage) search architecture connected to individual users, instead of the current link-topology-connected-to-no-one system weve all learned to love and hate.

Were also not ready to throw the keywordese out the door. We believe there is a lot of natural intelligence in keyword associations (i.e, John Battelles data base of intentions) that can provide an order-of-magnitude better relevance if they can be properly distilled  whether its applied to search, feeds, or media. We think the key is making it implicit (no explicit tagging or rating) and to make sure you have 100% participation  every user is both an information consumer and an information provider for every other user in the system.

[guyfrom2006]

Its time to roll on the next generation Search Engine

The popularity of the internet is that it offers a variety of content not available in any other medium. A search engine assists users to locate the information.

The Internet is supposed to be for fun not serious business.

And that is why most people use research tools and offline content to achieve their results. Internet search is the last place a true researcher would go to find top rated content. Only after all other options are closed.

What if the internet offered this as the very first option and people could be sure that all the information that the internet and internet search throws up is Top rated quality stuff and it is available free/subscribed/paid. I am myself a researcher and would love to have such a tool in my hands instead of having to spend a huge amount on buying proprietary research content.

Once I finished using the content, I need not pay for it. There are many websites such as Janes.com which provide this sort of quality info.

Present Internet Search is neither intelligent nor smart. Cluster search engines such as Vivisimo and clusty.com help to some extent but stull trash out the same stuff.

In this respect, I admire the efforts of NetAlter which is bringing a radical new search engine that would offer quality content and meta information. According to NetAlter, their search engine would offer a variety of pre-search and post search tools that enable sorting, comparing and analyzing of search results and also offer a single click ecommerce connection.

Check out the NetAlter search whitepaper and presentation.

<a class="jive-link-external" href="http://www.netalter.com/Solutions/search.pdf" target="_newWindow">http://www.netalter.com/Solutions/search.pdf</a>

<a class="jive-link-external" href="http://www.netalter.com/Solutions/Search.pps" target="_newWindow">http://www.netalter.com/Solutions/Search.pps</a>

[Himanshu_Joshi]

new search required

3D search idea by Micosoft is the one i find quite useful and it invloves user interaction as well... currently many search engine do give anonymous links to increase the number of pages but only some r useful.
thanx
Himanshu joshi
<a class="jive-link-external" href="http://montoojoshi.googlepages.com/onlinecompiler" target="_newWindow">http://montoojoshi.googlepages.com/onlinecompiler</a>

[readyforthefuture]

AI is already being added to search engines

People need to start following the AI bouncing ball at Google and other search innovators. The integration of the CYC taxonomy, for example, appears to be well under way with available frameworks that allow natural language and human logic to prevail over "keywordism" in the very near future.

<a class="jive-link-external" href="http://www.cyc.com/cyc/cycrandd/areasofrandd_dir/distributedai" target="_newWindow">http://www.cyc.com/cyc/cycrandd/areasofrandd_dir/distributedai</a>