Google tests: What a difference a few pixels make

These are two subtly different versions of the same Google page. One has slightly more white space around the first search result, giving it more prominence.

(Credit: Google)

In the old days, designers often had little more than gut checks and rules of thumb to determine the efficacy of creations such as advertisements or newspaper layouts. Later came expensive eye-tracking tests that showed how people scanned pages or computer screens. Now, though, Google, has the benefit of millions of users using its Web site to get things right.

The company, which in May began describing the split A/B tests it uses to see which of two alternative Google interfaces fares better, offered more details Tuesday. The theme: seemingly imperceptible differences are in fact perceptible.

"We test almost everything, even things that you would think are so small that we could not possibly care (nor could they possibly matter). In fact, small changes do matter, and we do care," said Ben Gomes, a Google distinguished engineer, in a blog post.

Google runs 50 to 200 tests at any given moment, he said.

Marissa Mayer, vice president of search products and user experience at Google, shows three slightly different versions of Google's search results page that the company tested with users during a May speech. The top, with the least white space, was more popular as measured by how much users searched.

(Credit: Stephen Shankland/CNET News)

Defining "better" can be tricky. Shifting elements to emphasize one can de-emphasize another, Gomes said. In one case, Google tried two versions of a "plus" box that, when clicked, shows a company's stock performance. The slightly bolder one yielded many more clicks. But is that a distraction or an improvement? Google hasn't decided which option is better overall, Gomes said.

Another example, vastly more obvious, is a test that lets users move individual search results up or down in the list.

"At this point, I can't say what we expect from this feature; we're just curious to see how it will be used," Gomes said.