Yelp's credibility problem: Blame it on algorithm?

Want to find a reputable dentist? How about a cozy bar for that perfect second date? A few years ago, you would have consulted friends. Today, chances are good that Yelp is the place you go.

But what if you found out that some of the reviews were written by hired hands--or that reviews were either removed or placed higher or lower on the page based on whether the merchant was willing to pay Yelp for special services?

Those are the allegations being made in an expose in the East Bay Express this week. The newspaper reported that six business owners said Yelp sales reps promised to remove or move negative reviews in exchange for advertising and that in six other instances positive reviews disappeared or negative ones appeared after owners declined ad deals.

The allegations could have consequences for Yelp, which was founded in July 2004 as a site where people can post reviews and ratings of businesses. It operates in about three dozen U.S. cities, as well as in Canada and the U.K.

In his rebuttal to the article posted on his blog, Yelp Chief Executive Jeremy Stoppelman denies all the allegations and questions the story's sources, including one named source he claims had reviews removed because they were deemed to be fake.

"There is irrefutable evidence that we do not do that," Stoppelman told CNET News on Thursday when asked whether the placement of some reviews is determined by advertising deals. "It's absolutely ridiculous that somebody would say we are going to write a review and call a business (to sell advertising). That's not how you build a sustainable business...Trust and integrity are key to staying in business."

The problem, according to Stoppelman, lies in the company's secret sauce for filtering out reviews.

At the mercy
Basically, merchants are at the mercy of a computer algorithm just like Web sites are at the mercy of what is known as the "Google Dance"--the monthly update of the Google search engine's index. One tweak of the Google index can potentially make or break a business.

According to Yelp spokeswoman Stephanie Ichinose, reviews are removed for one of three reasons:

• The consumer deleted the review or his/her account.

• Someone flagged the review as being in violation of the guidelines that ban reviews that are fake, written by a business' competitor, or based on hearsay.

• The review filter has detected some suspicious behavior that indicates a review is not legitimate.

It's normal for a merchant to see the number of its reviews rise or fall from day to day, and for reviews to mysteriously disappear, Stoppelman said.

"That's normal. It's part of how Yelp works," he said. "The reason we do that is to show only the most trusted content. It's nothing nefarious."

The timing of reviews that appear or disappear after a merchant accepts or declines an advertising contract is purely coincidental, Stoppelman said. He acknowledged that there can be false positives, in which legitimate reviews are removed.

"Sometimes we sacrifice legitimate content," he said. "The automated system protects both owners and consumers."

Getting to the bottom of the he-said-she-said behind the allegations would be difficult to do, particularly when dealing with a computer program that automates which reviews can stay and which must go. Reviews of Yelp on its own site are mixed, with some people complaining that their reviews were nixed for no apparent reason.

Trust issues
In an interview with CNET News, Yelp user "Estelle" said she has re-posted numerous times a negative review of a moving company she hired based on its positive Yelp reviews. Yelp kept removing her negative review with no explanation, she said. Three months after talking to an executive at the company about the problem, the review finally went back up, she said.

"I have switched to Yahoo Local for reading reviews," Estelle said. "I don't trust Yelp's tagline anymore, 'real people, real reviews.'"

Asked to explain why Estelle's negative reviews of the moving company were repeatedly removed, Ichinose said she could not go into specifics or risk revealing information that people could use to game the system.

One Yelp advertiser who asked to remain anonymous told CNET News that in his dealings with Yelp over the past few years, he had never been promised that his reviews would be manipulated. To the contrary...despite paying $750 a month for advertising services, he said, the site's refusal to remove several negative reviews that he felt were fake pushed his business from the top ranking to No. 7 in the search results, costing him a 25 percent drop in revenue.

"I wish I could pay more money to have the one-star reviews taken off," the advertiser said. "Yelp is remarkably unsympathetic to business owners who complain about negative reviews. They are tweaking the algorithm all the time, and they definitely don't give the business owner enough weight...They are being fair to the users, which is admirable. But as a business owner, it's really frustrating."

Merchants pay $300 to $1,000 a month for advertising services that deliver a specified number of impressions, or page views, said Ichinose. That can include a sponsored ad above the organic results that appear on a category search, as well as a photo slide show on the merchant's profile page and a review highlighted at the top of that page, she said. In addition, advertisers have the opportunity to place a small ad on the page of a rival business and block rivals from advertising on their page, she added.

Yelp discourages merchants from offering their customers compensation for writing positive reviews, but that is hard to enforce and likely happens. For instance, a colleague was offered a 10 percent discount to write a positive review of a UPS store in San Francisco on Thursday. There are also stories of "No Yelp" signs and other anti-Yelp sentiment on the part of companies that have been burned by negative reviews.

Keenly aware of PR problem
Yelp is keenly aware of its public relations problem and is hiring a manager of local business outreach to work with industry groups like restaurant associations and chambers of commerce, said Stoppelman. The site may also allow merchants to post public responses to negative reviews in the future, he said.

The company needs to address the situation--quickly. The perception of fraud can be as damaging to Yelp as actual fraud itself, said one business ethics expert.

"Somebody is lying, or there is just too much confusion. Either way, Yelp's got a problem," said Thomas White, director of the Center for Ethics and Business at Loyola Marymount University. "Trust is too easy to compromise and lose."

Kirk O. Hanson, an ethics professor and executive director of the Markkula Center for Applied Ethics at Santa Clara University, said Yelp needs to better understand and manage the ethics risk that its business model presents.

"Regardless of what the truth is about the accusations, Yelp has created a moral hazard which tempts their own people to manipulate things to increase revenue...They've created too many temptations for misbehavior," Hanson said. "Structurally, they have created incentives for their own organization to manipulate the postings."

Below are some of the mixed reviews that Yelp gets on its own Web site: