News Blog

What happens when you don't have good image processing in your search engine? You get inconsistent results, as seen in this page returned for the query "McDonalds".

(Credit: Google)

Google is starting to provide a fuller picture of the work it's undertaking to create a practical tool for image searches.

On its Google Research blog Thursday, the company offered a brief introduction to VisualRank, a system that sorts out images by means of visual cues rather than by text associated with the images. The write-up is a distillation of a much longer paper (PDF), "PageRank for Product Image Search," that two Google researchers presented last week at a conference in Beijing on Web technologies.

The VisualRank system is not yet live, and Google intimated that the image search technology would not become more widely available anytime soon. It did say that "in the coming months" it would offer more details on an "approach that has an easy integration with both text and visual clues."

In its initial VisualRank efforts, Google's research focused on product queries. That's in part because product queries correspond well to the type of "image features" that were central to the study. In addition, the company said, those types of queries are "popular in actual usage" and users have strong expectations about the results they expect, which gave the researchers key examples to address.

Google's VisualRank algorithm sorts images by how similar they appear to an inferred original.

(Credit: Google)

Google has also begun to broaden its initiative to take in other query types, including those related to travel.

As it moves forward, the search giant says it's exploring three main directions:

First, estimating similarity measures for all of the images on the Web is computationally expensive; approximations or alternative computations are needed. Second, we hope to evaluate our approach with respect to the large number of recently proposed alternative clustering methods. Third, many variations of PageRank can be used in quite interesting ways for image search. For example, we can use some of these previously published methods to reintroduce, in a meaningful manner, the textual information that the VisualRank algorithm removed.

Over the years, image search has been a significant challenge for Google and others, like start-ups Polar Rose and Riya, with most of the progress being in a fairly limited set of facial recognition characteristics. Last year, for instance, the company said that its Google Image Search could tell the difference between a picture with a face in it and a picture that lacked a face, though it couldn't distinguish between one face and another.

That sort of feature is increasingly common in digital cameras, which in some cases even recognize when a person is smiling or not. Camera makers are also working toward technology that knows who you photographed.

But recognizing a face or other object is a different order of business from delivering meaningful search results based on facial features or on object type.

Hewlett-Packard, meanwhile, has opened a lab at Tsinghua University in Beijing to delve into a wide range of media search types, including still images, video, and music.

Dust off your college calculus, because Google's image rank (IR) formula involves eigenvectors and iterative matrix multiplication.

(Credit: Google)

My company recently added a client for whom Like.com is a direct competitor.

The Web site was much-hyped and reviewed 10 months ago, when it fired up, including being dubbed the "First True Visual Image Search," but little has been made of it since.

If you believe the traffic trend data from Alexa, traffic to Like.com has mirrored interest by the media and blogosphere, having a spike at launch, followed by a marked decline.

Like.com employs the technology of Riya, an image search company that focused primarily--until Like.com became a factor--on facial recognition. Now that same recognition software is used to characterize fashion design and trends for the purpose of online shopping.

See a pair of pants or dress you like? Use Like.com's selection box to highlight the area of the apparel that catches your eye, and the engine will go out to find other pieces of clothing with similar features (at one point, this included photos of celebrities upon which to focus, but those, evidently, are no longer part of the site).

The question is, does this functionality have any real use, or is it just a neat Web application? Does it increase the likelihood of shoppers being paired with what they will buy, or is it simply another Web 2.0 exercise that will fall to the wayside with the other cool-but-useless applications out there?

Shopping for clothes is not a main focus in my life, but regardless, I decided to take two test runs on Like.com in an attempt to find something that I might at least consider buying.

First, I tried to find an item I'm familiar with by using the interface to guide me from the home page to the particular item. Second, I used Like.com in an attempt to match me with something I had no need for or knowledge of previous to the search engagement. The results were interesting.

In the first experiment, I tried to find a popular brand of pants I had tried on (and not purchased) in a store last weekend: a pair of Levi's Low Boot Cut 527 Jeans. Like.com has a photo of a pair of men's jeans on its home page, so I began there, highlighting the upper part of the photograph that most matched the Levi's. The "Likeness" menu popped up, giving me one of three choices to apply to this visual search: Color, Shape, or Both. I chose Both.

The results were wide-ranging, to say the least. With approximately 1,800 results returned to me, I was no closer to my particular Levi's, much less any Levi's. After scrolling through several pages of results (already outside of the image search boundaries), I finally came upon a pair of Levi's. I selected that pair of jeans, hoping to narrow my search.

At that point, however, I was no longer offered the visual search interface. Instead, I was taken to a purchase page at Amazon.com for that particular pair of jeans. Search over. Not even close. I found no way of using the visual-search interface to narrow my results.

The second time around, I looked for a pair of running shoes. I'm not sure what I want, in that regard, so I clicked on the Men's Shoes category and found a running shoe that looked good to me. It was a tennis shoe style, and I selected the toe of the shoe with the visual search interface. Lo and behold, I got a page full of tennis shoes to choose from. I clicked on a pair of Converse that caught my eye and, like last time, was taken outside of the Like.com Web site (to Zappos.com purchase page this time). I didn't buy the sneakers, but they did fulfill the general requirements of what I was looking for--I liked them!

With these experiences in mind, I have to say that as a pure form of search, this type of visual search is extremely limited. Like.com could certainly make improvements to its ability to narrow a search using the tools, but even then, I suspect that sooner rather than later, a wall would be found.

While visual search is useful in some shopping searches--browsing searches as opposed to specific searches--it is only in tandem with traditional textual search that the tool has any real use. Still, it's an interface that could grow beyond a "cool app," especially if it is branded as just one search tool among many, rather than as a destination in its own right.

It could make for a nice complement to Google's search tool set; perhaps Google should acquire it?

PHOENIX--One of the hottest companies you may not have heard of yet is Redwood City, Calif.-based Riya.

Essentially, Riya's Web-based service involves photo search. But that oversimplifies what could be one of the coolest--or scariest--innovations to come along in some time.

The idea is based around trying to bring order to the thousands of unnamed digital photos we all have on our hard drives.

Riya asks users to upload sets of photos and then add names to faces the service doesn't recognize. Once you add a name once, the software is designed to recognize each and every instance of that person in the photo library, even if their face is just in a framed photograph on a wall in the background.

The software can also recognize text--from street signs to name tags.

Some have worried that Riya invades privacy because it asks you to provide e-mail addresses of other people you know to confirm if they've tagged in the same way faces that appear in your photos. But the company has said it has no ties to the government and just wants to help people sift through their mountains of pictures.

Well, you decide for yourself.

In the meantime, there have been rumors that Google was buying Riya, but the company's CEO, Munjal Shah, denies it. Still, he hinted at Demo '06 here that Google may well have been one of the investors who recently put $15 million into Riya. If so, we could expect to see the software incorporated into Picasa sometime soon.