Webware

There is something about the Semantic Web that gets me all excited about the future of the Internet. Services like Glue are making sense of data all across the Web and allowing users to interact with it.

CNET News' Dan Farber has covered the basic functionality of Glue, but to recap, after installing the Glue Firefox extension, when you visit supported sites about movies, music, books, or other included categories, the Glue bar will slide down, giving you the opportunity to give your input on the product.

One of the things Glue is introducing today is the ability to interact with a piece of content and have a discussion across multiple sites. For example, if I visit the Wikipedia page on the film "Slumdog Millionaire" and write a review of it on Glue, my friends see my review when they visit the "Slumdog Millionaire" page on any of the supported sites, including Amazon.com, Fandango, IMDb, Metacritic, Netflix, and many others.

The opinions of your friends on any particular piece of content just appear as you browse. Today, Glue is releasing the latest version of its service, featuring improvements in content recommendation, Facebook Connect integration, and the improvements to conversations noted above.

In this release, Glue has included support for Facebook Connect. This is going to be a big step forward for the service. The main challenge that new users face when signing up for a new social network is finding their friends. Now Glue users can automatically grab their social graph from Facebook and interact with their friends in Glue.

The "connected conversations" feature that I talked about briefly a couple of paragraphs above is probably the most impressive aspect of this release. Having all of your friends' opinions on a particular product, combined so that you can view them, regardless of what site you choose to browse for the content on, is very powerful.

Glue almost completely eliminates the friction involved with getting your friends' opinions on whether you should buy, rent, or watch that movie you've been eying. The Glue bar slides down at just the right moment when you are wondering what your friends think about the movie and happily reports their feedback.

Glue has also added some nice recommendation features. Glue now breaks down what content is popular in the various categories within your circle of friends and in the entire Glue population. This should help with some additional content discovery and prove to be interesting in trend monitoring. Maybe the "worldwide top lists" can fill the void left by Facebook Pulse in keeping track of what's hot.

This release of Glue is definitely a solid offering, and I think that the additions of Facebook Connect and connected conversations make it a really killer product. The best part of the product is that it doesn't bug you on a constant basis, but it provides useful information on your friends' opinions when you are looking for it.

Being a fairly painless Firefox Extension install, Glue is worthy of at least a test run. Load up your Facebook friends, browse around, and let us know what you think of Glue.

SUNNYVALE, Calif.--Call it SearchMonkey Lite--an easier way for a Web site to spotlight its videos, games, and documents in Yahoo's search results.

Yahoo has been working to let publishers spotlight their content in its search results through a program called SearchMonkey, but the company has concluded the technology's power comes at the expense of ease of use. Now Yahoo is offering a lightweight way to use SearchMonkey that it hopes will make the service approachable to average Web page creators.

The company posted a blog entry with some basic text that can be tweaked then inserted into Web pages. Doing so will mean Yahoo's Web crawling software will recognize videos, games, and documents, and those data types then can be shown prominently alongside the Web address in Yahoo's search results, said Tom Chi, senior director of product for Yahoo search in an interview here at Yahoo headquarters.

"There's very little code required to engage with this," Chi said of the templates Yahoo is offering. "Adding that extra bit of structure helps those who might be less technically experienced."

Video results are appearing now, and games and documents should start appearing over the next couple weeks, he said. However, Yahoo will add the results in gradually to ensure its results aren't being gamed or polluted with spam, he added.

Yahoo is trying to make its search more useful and therefore more used, part of its attempt to compete with search leader Google. The more search results are shown, the more opportunities the search provider has to show related advertising.

With this lightweight version of SearchMonkey, search results become more of a destination unto themselves. Right on the search page, the videos can be watched, the games can be played, and the documents can be read. (See screenshot below.)

SearchMonkey relies on Yahoo's search engine finding "structured" data on the Internet--Web sites whose elements have been labeled so computers can know when they've found an address, a video, or other particular types of information. That structured data is a crucial element of what's called the Semantic Web, a years-old idea that computers should be able to understand the meaning and not just the text of Web sites.

"We hope that through programs like this, it'll be possible for publishers to start getting engaged with the Semantic Web," Chi said.

When Yahoo announced BOSS (Build Your Own Search Service) in 2008, the company said it planned to make money from the service. On Wednesday, though, the company announced it's got a new way in mind: charging for high-volume use of the search data.

Yahoo will charge for use of the BOSS API (application programming interface), the service by which other Web sites can extract Yahoo's search data then repurpose it to their hearts' content, according to a blog post by Ashim Chhabra of Yahoo's Search BOSS team. Previously, the company had planned to make money from BOSS by requiring outsiders with high-traffic sites to show Yahoo search ads next to their results.

The new approach allows companies to pursue their own monetization strategies and will help make the API itself more useful by lifting constraints, Chhabra said.

"We're introducing fees for a couple of reasons. First and most importantly, we're hard at work on a number of technologies that will enhance both the functionality and performance of BOSS, and usage fees will help support this development," Chhabra said. "Second, we believe that introducing the proposed pricing structure will improve the ecosystem by optimizing capacity for our serious developers."

BOSS is one part of Yahoo's attempt to make its search more competitive with dominant rival Google, which gained market share over rivals in January, according to Nielsen Online.

One limit that's lifted will be the amount of search results that can be retrieved with one call to the BOSS API; with the fee structure, that limit goes from 50 to 1,000. Yahoo also will offer a service level agreement (SLA) so outsiders can count on BOSS working.

The new fees likely will go into effect late in the second quarter, according to the BOSS fee page; those who use the service will pay on the basis of 10-cent units. For example, retrieving the first 100 results for 1,000 searches costs 10 units; developers will get 30 free credits a day, and the rate goes down during off-peak hours.

SearchMonkey injection
Yahoo also announced it's grafting some SearchMonkey technology into BOSS. SearchMonkey can gussy up certain Yahoo search results in cases when the Web sites listed describe their own data with computer-oriented descriptions called microformats such as a restaurant indicating its address. This idea, called the "semantic Web" and long under development, theoretically gives computers a better understanding of what's on Web pages.

The BOSS API now can be set so that search data it retrieves spotlights that structured data, Chhabra said.

BOSS now also shows two other elements: longer 300-character descriptions of each page in search results, up from 170 characters, and some data retrieved by Yahoo's SiteExplorer technology, which can show details such as popular pages within a particular Web site or a list of other Web sites that link to it.

For the last few years AdaptiveBlue has offered a semantically rich Web application that understands things such as books, movies, and music. Clicking on text, such as a company or movie name, brings up a context-sensitive menu of related links. The company is taking its technology a step further, adding a social dimension and renaming the product, "Glue." Along with Radar Networks' Twine and Powerset's Wikipedia search engine (acquired by Microsoft), Glue offers a compelling glimpse into how the Semantic Web will add a new, powerful level of intelligence to the Internet.

Rather than just connect things to related data and services, it also connects things to people and people to people and their things. For example, when a Glue user visits a site with things the software recognizes, such as a movie, artist, wine book, restaurant, or stock quote, a bar appears at the top of the screen with a list of friends and other people in the Glue network who looked at that object. Users can leave brief comments to share an opinion with others.

"Glue works as a contextual filter," said Alex Iskold, founder and CEO of AdaptiveBlue. "We show relevant information from friends about the things they visit. They don't have to sift through lengthy lifestreams. For example, if you have 100 friends in FriendFeed, you are a human filter trying to sift through it and the information is completely out of context. The idea is to get the useful information 'chunked' contextually on the pages you visit. We are not asking people to change their habits."

The people surfaced in the Glue bar could have seen the object, such as a movie title, on a variety of sites. "People look at movies at different times and places, but the core semantic technology can understand the same thing and correlate it. As a movie fan, you just want to know what your friends think. It doesn't matter when or where the user visits things; Glue automatically connects them. There is no Glue destination site--the network is the user's context across the Web," Iskold said.

Glue also taps into existing social networks, such as Facebook and Twitter, to add friends, or to "follow" other people. The Glue Navigator allows users to browse the network of people and things, and what friends have identified as a "like" and what they have to say about objects. Glue can display all the music that a friend has viewed and drill down, offering contextual shortcuts to find out more, such as reviews and shopping links, about things on the Web. Glue remembers only the last 20 last things visited, and the things "liked" or commented upon.

Each user has a profile page that shows likes and the number of followers and who the user is following. "It's a way of cross-pollinating interests. You can see what I am interested in and perhaps it is the same books or wine with which you have an interest," Iskold said. "Glue also allows you to claim pages that represent you, such as a blog, FriendFeed, or Twitter. It's an outlet where people know where to find and connect with you. For example, other Glue users could see what you are up to recently on your personal blog."

Glue impressed investors at RRE Ventures and Union Square Ventures (Series A Lead) enough to fund a $4.5 million series B round recently. The company has a good chance of making it through the meltdown.

In the midst of the financial meltdown and a contentious upcoming election, you might think the U.S. government and taxpayers are just funding wars, bank bailouts, and bridges to nowhere or somewhere. But this is the same government that funded the Internet way back when and is also funding the next generation of technologies that will make the current Internet seem like a Model-T.

Over the last several years, the U.S. government--via DARPA (Defense Advanced Research Projects Agency) grants--has invested hundreds of millions of dollars in PAL, an acronym for "Personalized Assistant that Learns." Smarter software and networks and augmenting human intelligence are useful in times of war and peace.

As part of the PAL project, more than $200 million of DARPA money has been poured into CALO (Cognitive Assistant that Learns and Organizes) over the last five years. CALO has been run out of SRI International with the assistance of 25 research organizations and 400 researchers.

Several companies, including Radar Networks, Farecast (acquired by Microsoft) and Adapx, have been spun out of SRI based on some facet of CALO technology. The latest, Siri, was founded in December last year and has raised $8.5 million in series A funding from Menlo Ventures and Morgenthaler Ventures.

At this point, Siri's management is being secretive about what the company is developing. The elevator pitch goes something like, "Users' online lives are becoming more complicated and getting out of control for mainstream users. What if there was an easy way for normal users (non-power users) to ask the Internet to help them."

According to the Siri PR pitch, the product is "a new interaction paradigm for the consumer Internet experience that applies intelligence at the interface." The company expects to release a beta version of its initial product in the first half of 2009, according to Dag Kittlaus, a former Telenor Mobile and Motorola executive who is a co-founder and CEO of the company.

"We have to be careful at this stage," Kittlaus told me. "We don't like to play these games, but we need to keep a tight lid on what we are specifically doing. We have some original ideas of what the product is going to do, but we don't want to spark ideas among potential competitors." Those competitors would likely be masters of the Internet with large Internet footprints and research prowess like Google, Microsoft, and Yahoo.

Kittlaus did allow that Siri has more than a dozen partners, presumably large, well-established distribution players that can help build a consumer market for Siri's product. Unlike most Web start-ups, Siri has a business model, Kittlaus claimed. "We have good business models, both existing and emerging. We think CPA (cost per action) is the future, and this specific application is good for CPA and we are partnering on that."

He also touted the pedigree of the company's current cadre of 19 employees. "They are mostly engineers from Yahoo, Google, SRI, NASA, and Xerox PARC," he said. The chief architect of the CALO project, Adam Cheyer is a co-founder and vice president of engineering at Siri, and Tom Gruber, a well-known artificial intelligence and semantic Web expert, is a co-founder and CTO.

Cheyer described CALO as superset of what Siri is developing. "The CALO project is building an automated assistant to help manage and improve your life. The technology spans all aspects of interaction--natural language processing, speech recognition, and planning and reasoning capabilities--and interfaces with all kinds of systems, such as email and contacts," he said.

"Learning in the wild is core focus," he continued. "We want it to improve over time and learn from users with no coaching and without changing any code. We are taking the key elements from the project to commercialize it in a form that will delight users. We are not building systems that do things but that learn how to do things."

CALO sounds like a representation of the famous Apple Knowledge Navigator video from 1987.

"Siri is a subset of that concept," Cheyer said. "We have to keep in mind existing user behavior. It will feel like something close to what people use a lot. We will add speech recognition and other features as we go. We don't want to take such a leap that people cannot identify with it. We'll do things similar to but more advanced than what we do now. The longer term vision is the Knowledge Navigator, although it is an early chapter now and it might look different than that."

According to Gruber, intelligence at the interface allows the computers to make recommendations, like a personal assistant:

The interfaces we use to interact with the world's information are getting smarter. Web portals gave us someone else's idea of the content we should see. Then came search engines, which let us tell the system what we want, one query at a time. We are about to see the next wave -- intelligence at the interface -- in which the system knows about us, our information, and our physical environment. With knowledge about our context, an intelligent system can make recommendations and act on our behalf.

Siri may be working on more intelligent Web interfaces that can make inferences based a wide variety of user activities (the "lifestream"), learning over time on its own, and then taking actions on behalf of users. For example, if you are booking travel or looking for a restaurant, Siri would know your preferences and about travel sites or restaurants, integrating data and context from multiple sources to deliver personal assistance. This could be especially useful in mobile scenarios where you don't want to wade through pages of search results or deal with complex interactions.

We'll have to wait for next year, if the company stays on schedule, to see whether Siri can really define a new paradigm for experiencing the Web.

Two companies recently pitched me on their semantic engines. These are not search engines, which is what most people think. Rather, they are databases and algorithms that hold the structure of language (in both cases, the English language). At the most basic level semantic engines tell you what's synonymous with what. At the advanced end of the spectrum they know how grammatically similar phrases like "take a seat," "take a stand," and "take a lollipop," mean completely different things.

These engines can be used by search products to greatly improve results. Powerset, now a part of Microsoft, made a big deal of its semantic chops by showing how vaguely worded search queries would return just the results you wanted. Now, it seems, that raw semantic technology is about to become mainstream.

Cognition recently announced its "world's largest semantic map of the English language," sporting more than "10 million semantic connections." The company is rolling the technology into products like CognitionSearch for the Enterprise, which is a knowledge mining tool, as well as an "eDiscovery" product for the legal industry that enables lawyers to "quickly and efficiently find incriminating, smoking gun documents." The company is also applying its technology to a new advertising engine.

The much smaller and newer company, Eeggi, which I was introduced to at Web 2.0 Expo in New York, is also building an engine for discerning meaning. Founder and chief scientist Frank Bandach told me his model was mathematical (his training is as a prime number theorist) and that his engine goes well beyond understanding synonyms. In his demo, he entered the query "Mary kissed John," and showed how traditional word-matching engines picked up pages there were also about John kissing Mary. His system understands English well enough to filter those out as misses.

Bandach says that he's got most of the English language in his system, and that he did English first, "because it's hard. Only Finnish is harder." He's going to work on German next, by feeding it some German dictionaries, which sounds like a science-fiction way to seed a semantic engine, but he said it's enough to get the system going. Bandach says his algorithms are efficient and not, like Powerset's, CPU hogs.

Unlike Cognition, Eeggi is an early-stage project with only four people working on it. It's far too early to tell if the technology is robust and scalable enough to compete with Cognition or Powerset. But I am encouraged to see small companies working on this problem and claiming intellectual breakthroughs. I really would not be surprised to see "meaning engines" become available to Web developers in the same way spelling checkers and grammar engines are now. I have no idea what developers will build with this technology, but I can't wait to see it.

See also: Cycorp.

Click here for full coverage of Web 2.0 Expo

As expected (see previous reports), Microsoft scooped up Powerset to buttress its search efforts.

It's not a replacement for increasing market share by acquiring Yahoo Search, but it gives Microsoft some differentiated search technology and top engineers for less than $100 million. Ramez Naam, group program manager of Live Search, said the Powersoft negotiations happened in parallel with the Yahoo talks over the last few months. Google and Yahoo may also have been interested in Powerset, but no one is talking.

Whether Microsoft can leapfrog Google over the long term with this semantic engine remains to be seen.

Powerset had done a good job of creating a rich semantic layer on top of Wikipedia, but bringing natural language and slick semantic-based interfaces to the entire Web is a long-term and very costly endeavor.

"With an existing search infrastructure, incredible capital resources, unlimited data, a leading search team, and clear mission to revolutionize the search landscape, Microsoft can rapidly accelerate our progress in building semantic search technology and bringing it to full Web scale," Powerset's Mark Johnson said in a blog post about the acquisition.

According to a blog post from Satya Nadella, Microsoft's senior vice president of Search, Portal, and Advertising, Powerset's engineers will join the Search Relevance team and remain in San Francisco.

Back to the leapfrogging Google question. Much of what Powerset has enabled with its technology is a superior user experience for searching. Powerset's Wikipedia search, which surfaces concepts, meanings, and relationships (like subject, verbs, and objects in a language), is the very small tip of the iceberg.

If Microsoft can succeed in extending Powerset's technology to key parts of the Web corpus, Google will have to figure out a way to match the quality and user experience. And, there is little doubt that if Google decided that what Powerset and Microsoft are doing as one is important, the company dedicated to dominating search through its engineering prowess will circle the wagons.

A few months ago, Powerset co-founder and CTO Barney Pell told me that his start-up company's software was a first step in changing the way users search and consume Web content. "It's a complete shift. You see this and you want to experience all content in this way. And, as an introduction, it will drive huge investment in semantic and linguistic technology, just as investments were made in information retrieval and scalable databases in the past," he said.

During a conversation after the announcement, Pell told me, "Natural language search will be the center of innovation for the next 20 years." It will likely take 20 years to engineer the semantic, natural language Web that Tim Berners-Lee envisioned in his 2001 essay in Scientific American.

Distributed commenter reputation service SezWho is growing a little bigger Wednesday morning with the acquisition of Tejit, an engine that tracks content around the Web to see how it links up with people, events, places, and more. The tool began as a pet project for creator Indus Khaitan, who wanted to sort through blog content to find who had the most gravitas on any topic or in a certain field.

What does this mean for SezWho? SezWho's founder, Jitendra Gupta, tells me that in the next two to three months users of the add-on reputation system for comments, forums, and more will get a whole new layer of reputation ratings based on what types of communities they're interacting with. For example, a highly reputable user who frequents tech blogs will come into blogs in other disciplines like science or politics with a slightly higher starting reputation than a standard user. The same goes for bad users, who will come into new areas with a bit of a warning above their heads to give other users the heads-up.

This same system will be applied retroactively to any other conversations that have been tracked for SezWho users. That will effectively augment some people's ratings for better or worse.

In my call with Gupta last week, he spent a lot of time digging into FriendFeed, which has received a lion's share of the attention in the Web 2.0 community and quickly is becoming a destination site of its own.

Gupta argues that SezWho is laying a better groundwork for content owners since people will follow what their friends are doing online using a system that doesn't take eyeballs away from the original content--something FriendFeed has recently strayed away from a bit with its rooms implementation. I don't put the two in the same class of sites, but I can see where he's coming from after designing a service that lets you keep tabs on people as long as they're visiting SezWho-enabled sites.

As to whether there would be a client-side version of the reputation service to remedy interactions with non-integrated sites, Gupta would not comment.

Amid speculation that Microsoft is looking to make an acquisition, Powerset launched a public beta of its Wikipedia search engine. It brings a new, rich semantic dimension via natural language query processing to Wikipedia that greatly improves the search and reading experience.

The company calls it a first step in changing the way users search and consume Web content. "It's a complete shift. You see this and you want to experience all content in this way," Barney Pell, co-founder and CTO of Powerset, told me. "And, as an introduction, it will drive huge investment in semantic and linguistic technology, just as investments were made in information retrieval and scalable databases in the past. People working in this space will be very marketable."

Powerset's natural language search technology is based on patents licensed exclusively from PARC and its own proprietary indexing. Powerset's engine has read 2.5 million Wikipedia pages and extracted "meaning" from the sentences, creating a navigation and semantic layer on top of the popular Web encyclopedia. Following is a pictorial tour of Powerset features:

Powerset's Wikipedia search engine isn't going to slow down the Google in the near term, but it will raise the bar on the search experience for all players. "There are implications beyond Wikipedia," Pell said. " Search is not done. You can see the emerging Semantic Web with our integration of Wikipedia and Freebase. We will add other components with structured data and ways to answers questions."

Powerset has said that the longer term plan is to read, linguistically analyze and index 20 billion documents on the Web, which will be a costly and ambitious undertaking. (Getting acquired by Microsoft would be helpful for that project. Powerset has received $12.5 million in Series A funding from Foundation Capital, Founders Fund, and angel investors in 2006.)