Web design not what you pay for

When it comes to Web design, you may not get what you pay for.

That was the lesson hundreds of Web developers came away with after User Interface Engineering analyst Jared Spool yesterday presented findings on a Web design and information retrieval study at the Web Builder conference in San Francisco.

In a test of ten Web sites conducted last year, each with at least 3,000 pages, study participants rated a site that reportedly cost $300 million the worst, and a $10,000 site the best.

The bad news: When it came to information retrieval, even the best of the Web sites were bad. On a scale of one to ten, none rated higher than a 4.5.

The nine sites surveyed included those of industry heavyweights Hewlett-Packard, Disney (which produced the site that testers said cost $300 million, a figure Disney disputes), and NEWS.COM publisher and Web Builder conference organizer CNET.

The trouble with these popular destinations on the information superhighway, according to the study, was that they make it too difficult to find information.

More coverage on CNET Radio

Researchers tested more than 50 users--all of whom had at least some Web experience--by asking them to find specific information at particular Web sites in what Spool termed an "online scavenger hunt." The participants' activities were monitored for three hours. The users encountered daunting stumbling blocks.

One problem was that links were not sufficiently descriptive. In an example from the Disney site, a text link reading 101 Dalmatians under the subject heading "Movies" might have led to anything from a description of the movie to a lengthy download of the movie itself.

Disney noted that its site had undergone extensive renovations since the study. The company also disputed Spool's contention that the company's Web site had cost $300 million.

"That number is definitely grossly overstated," said Disney spokesperson Diane Passarelli. But Passarelli declined to disclose how much money Disney had spent on the site and its relaunches, or how far off the UIE number was.

Spool later defended the figure, which he characterized as an estimate based on the reports of engineers who worked on the project and on the public statements of Disney executives.

In contrast to Disney's multimillion-dollar site, the $10,000 site for Edmunds's Automobile Buyer's Guides, which also has been redesigned since the study was conducted, offered unambiguously named links such as "Click here for Car Prices!"

Even if users can't find what they're looking for through well-labeled links, many companies offer site searching as a shortcut to information. But the study found that site searching was not only useless but also detrimental for information gathering: users were 50 percent more likely to find what they were looking for if they never hit the search button.

Part of the trouble with site searching was that some sites had multiple search engines that were insufficiently differentiated from one another.

Another problem was that search results were long lists of titles and file names that did little to describe their documents' contents.

Web designers aiming for clarity with lots of white space and sparse text are defeating their purposes, according to the report. In what might seem a paradoxical result, study users said more white space caused sites to be too complicated, over-detailed, visually confusing, unclear, and "not enticing." Mistrustful of the results, the researchers tested the effects of white space five different ways, only to come up with similar results.

Another surprising finding: users find that the less "readable" a page is, the more authoritative, clear, and useful it is.

The reason is that readability is concerned with the structure of sentences, the length of words, and other measures of good English prose. But Web surfers don't read, concluded the researchers; instead, they skim, making traditional measures of readability irrelevant to Web information retrieval.

Another surprising finding for some Web designers was that users found animation and movement irritating, sometimes to the point that they would cover up the offending .gif with their hand.

This observation seemed to clash with studies showing that animation results in twice the click-through rate for Web banners. But the apparent contradiction stems from two disparate groups of Web users: those looking for specific information and those randomly surfing.

"Surfing and information retrieval are two different things," Spool noted. "You need to know what you are designing for."

Web developers agreed.

"This is all pretty common sense advice," said Teri Olsen, Web coordinator for the University of Utah. "It reinforced stuff that those of us who aren't doing cutting-edge design already know, in terms of things like descriptive links and not using animated .gifs."

Olsen did encounter some surprises, however.

"It was very interesting to see how the average user responded to white space and site searching," Olsen said. "I'm definitely going to go back and take a look at how search is serving our users."

Another Web developer applauded the study, but said that in testing average users it wasn't necessarily measuring the Web's ultimate audience.

"A lot of what the study found is partly a measure of the youth of the medium," said Scott Jarol, senior project manager for MediaLive's SurfMonkey site. "When people get used to the concept of hierarchical categories, some of the things he said were obstacles will turn out to be beneficial."