Yelp's credibility problem: Blame it on algorithm?
Want to find a reputable dentist? How about a cozy bar for that perfect second date? A few years ago, you would have consulted friends. Today, chances are good that Yelp is the place you go.
But what if you found out that some of the reviews were written by hired hands--or that reviews were either removed or placed higher or lower on the page based on whether the merchant was willing to pay Yelp for special services?
Those are the allegations being made in an expose in the East Bay Express this week. The newspaper reported that six business owners said Yelp sales reps promised to remove or move negative reviews in exchange for advertising and that in six other instances positive reviews disappeared or negative ones appeared after owners declined ad deals.
The allegations could have consequences for Yelp, which was founded in July 2004 as a site where people can post reviews and ratings of businesses. It operates in about three dozen U.S. cities, as well as in Canada and the U.K.
In his rebuttal to the article posted on his blog, Yelp Chief Executive Jeremy Stoppelman denies all the allegations and questions the story's sources, including one named source he claims had reviews removed because they were deemed to be fake.
"There is irrefutable evidence that we do not do that," Stoppelman told CNET News on Thursday when asked whether the placement of some reviews is determined by advertising deals. "It's absolutely ridiculous that somebody would say we are going to write a review and call a business (to sell advertising). That's not how you build a sustainable business...Trust and integrity are key to staying in business."
The problem, according to Stoppelman, lies in the company's secret sauce for filtering out reviews.
At the mercy
Basically, merchants are at the mercy of a computer algorithm just like Web sites are at the mercy of what is known as the "Google Dance"--the monthly update of the Google search engine's index. One tweak of the Google index can potentially make or break a business.
According to Yelp spokeswoman Stephanie Ichinose, reviews are removed for one of three reasons:
• The consumer deleted the review or his/her account.
Jeremy Stoppelman,
Yelp CEO
• Someone flagged the review as being in violation of the guidelines that ban reviews that are fake, written by a business' competitor, or based on hearsay.
• The review filter has detected some suspicious behavior that indicates a review is not legitimate.
It's normal for a merchant to see the number of its reviews rise or fall from day to day, and for reviews to mysteriously disappear, Stoppelman said.
"That's normal. It's part of how Yelp works," he said. "The reason we do that is to show only the most trusted content. It's nothing nefarious."
The timing of reviews that appear or disappear after a merchant accepts or declines an advertising contract is purely coincidental, Stoppelman said. He acknowledged that there can be false positives, in which legitimate reviews are removed.
"Sometimes we sacrifice legitimate content," he said. "The automated system protects both owners and consumers."
Getting to the bottom of the he-said-she-said behind the allegations would be difficult to do, particularly when dealing with a computer program that automates which reviews can stay and which must go. Reviews of Yelp on its own site are mixed, with some people complaining that their reviews were nixed for no apparent reason.
Trust issues
In an interview with CNET News, Yelp user "Estelle" said she has re-posted numerous times a negative review of a moving company she hired based on its positive Yelp reviews. Yelp kept removing her negative review with no explanation, she said. Three months after talking to an executive at the company about the problem, the review finally went back up, she said.
"I have switched to Yahoo Local for reading reviews," Estelle said. "I don't trust Yelp's tagline anymore, 'real people, real reviews.'"
Asked to explain why Estelle's negative reviews of the moving company were repeatedly removed, Ichinose said she could not go into specifics or risk revealing information that people could use to game the system.
One Yelp advertiser who asked to remain anonymous told CNET News that in his dealings with Yelp over the past few years, he had never been promised that his reviews would be manipulated. To the contrary...despite paying $750 a month for advertising services, he said, the site's refusal to remove several negative reviews that he felt were fake pushed his business from the top ranking to No. 7 in the search results, costing him a 25 percent drop in revenue.
"I wish I could pay more money to have the one-star reviews taken off," the advertiser said. "Yelp is remarkably unsympathetic to business owners who complain about negative reviews. They are tweaking the algorithm all the time, and they definitely don't give the business owner enough weight...They are being fair to the users, which is admirable. But as a business owner, it's really frustrating."
Merchants pay $300 to $1,000 a month for advertising services that deliver a specified number of impressions, or page views, said Ichinose. That can include a sponsored ad above the organic results that appear on a category search, as well as a photo slide show on the merchant's profile page and a review highlighted at the top of that page, she said. In addition, advertisers have the opportunity to place a small ad on the page of a rival business and block rivals from advertising on their page, she added.
Yelp discourages merchants from offering their customers compensation for writing positive reviews, but that is hard to enforce and likely happens. For instance, a colleague was offered a 10 percent discount to write a positive review of a UPS store in San Francisco on Thursday. There are also stories of "No Yelp" signs and other anti-Yelp sentiment on the part of companies that have been burned by negative reviews.
Keenly aware of PR problem
Yelp is keenly aware of its public relations problem and is hiring a manager of local business outreach to work with industry groups like restaurant associations and chambers of commerce, said Stoppelman. The site may also allow merchants to post public responses to negative reviews in the future, he said.
The company needs to address the situation--quickly. The perception of fraud can be as damaging to Yelp as actual fraud itself, said one business ethics expert.
"Somebody is lying, or there is just too much confusion. Either way, Yelp's got a problem," said Thomas White, director of the Center for Ethics and Business at Loyola Marymount University. "Trust is too easy to compromise and lose."
Kirk O. Hanson, an ethics professor and executive director of the Markkula Center for Applied Ethics at Santa Clara University, said Yelp needs to better understand and manage the ethics risk that its business model presents.
"Regardless of what the truth is about the accusations, Yelp has created a moral hazard which tempts their own people to manipulate things to increase revenue...They've created too many temptations for misbehavior," Hanson said. "Structurally, they have created incentives for their own organization to manipulate the postings."
Below are some of the mixed reviews that Yelp gets on its own Web site:
(Credit:
Yelp)
Elinor Mills covers Internet security and privacy. She joined CNET News in 2005 after working as a foreign correspondent for Reuters in Portugal and writing for The Industry Standard, the IDG News Service, and the Associated Press. E-mail Elinor. 
I would feel much more comfortable if yelp wasn't "the" review website and it had at least one other major competitor. This way yelp wouldnt have too much pull and in effect level the field a bit.
http://www.yelp.com/topic/san-francisco-disappearing-reviews-2
There is tons of this going on and it is very suspicious.
/P
I stopped using CitySearch in 2001 when it was so painfully apparent that fraudulent reviews had overrun the listings. CitySearch, unfortunately, became inordinately dependent on their revenue model and had no incentive to validate their reviews.
Professor Hanson neglects to mention that Moral Hazard is a fact of life in almost all business models -- Google's included. What matters is whether the company falls prey to said hazard. As a friend to and user of Yelp, it is my honest opinion that Yelp's management team and employees exhibit the same ethics as Google's founders. Both companies are complex and opaque, but both are committed to their users above all else.
I could create a page on a subject matter only I know of that nobody else in the entire world knows of and on a daily basis as I edit it then near completion of the edit someone sees my Wiki entry and erases the entire thing or deletes teh entire subject for no reason based on fact and I have ZERO recourse other than wishing I could harm their NIC card.
If someone can find a PERFECT way to easily filter out fakes and spams from true and trust, we should nominate him/her to solve the current economic crisis.
I met Jeremy a few years ago and he told me that this was what they did. I said at the time that I thought it was unethical. Him and his crew laughed about it. This way before Yelp had become popular.
After declining to advertise and noticing that our positive reviews started to disappear, I started printing all of our reviews from time to time. This article inspired me to review those archives and, in less than an hour?s time, I have already found four positive reviews that were suppressed. Using the ?member search? feature on yelp, I was able to locate the reviewer (proving the account has not been deleted) and find the original review (proving that the review was suppressed by yelp, not deleted by the user). There is nothing ?suspicious? about any of these reviews, other than the fact that they have been mysteriously suppressed by yelp.
When our business reached 100 reviews, one of our positive reviews would drop off, bringing our total number of reviews back to 99. That happened five times over a period of three weeks, until yelp apparently gave in and allowed us to go over 100 reviews. Stoppelman blames anything suspicious about yelp?s business tactics on their so-called ?spam algorithm?, but there is no way that five long-standing positive reviews would suddenly appear to be ?spam?. Yelp is simply using spam, something most people despise, as an excuse for manipulating its content to its own advantage. Yelp is insulting the intelligence of business owners and reviewers by suggesting that spam is a complicated problem that we just don?t understand. Forcing users to verify an email address prior to posting a review and allowing users to flag posts is sufficient for fighting spam. I think the only thing complicated about yelp?s algorithm is coming up with a good explanation of how it works without revealing unethical business practices.
I also have a documented example of retaliation. I flagged a review from a user whose post was nothing more than a personal attack and did not include an actual review of the business, two things identified in yelp?s terms of service as inappropriate behavior. Yelp ignored my flag, letting the inappropriate review stand, and instead removed my own review. I took a screenshot before flagging the item, captured the contents of the flag itself, and then took a screenshot showing that my review was missing (not deleted, just suppressed or hidden from view, which a user cannot do). Yelp apparently prefers to suppress reviews, rather than delete them, because they can continue to use them in their marketing numbers. Yelp claims that more than 4.5 million reviews have been written so far, but it does not disclose how many are suppressed from view or written by paid staff.
Yelp has some serious explaining to do if they expect to recover from this public outing of their wrongdoing. Without the documented evidence I?ve described above, they will just continue to dismiss this problem as ?confusion? on the part of small business owners. Many small businesses feel that yelp has great power over them, and that?s exactly what yelp wants. If you?re scared, you won?t put your business at risk from retaliation and you won?t provide the evidence that is needed to put a stop to yelp?s unethical business practices once and for all. I believe that a class action lawsuit will take shape once documented evidence is provided and more coverage in major newspapers occurs. I plan to send my evidence to all papers that have already reported on the matter and those that probably should, including the following: New York Times, Wall Street Journal, San Francisco Chronicle, San Jose Mercury, Chicago Tribune, Dallas News, London Times, East Bay Express, Daily Cal, and any other local papers that can help to spread the word about yelp?s unethical business practices. I hope other business owners will do the same and help keep this great momentum going until a solution can be found.
They don't identify these Scouts though they post large numbers of reviews while on the clock.
I would consider this an ethical issue as well.
anonybot aka former yelp scout
If Yelp loses credibility, his wife could lose her job, and he loses income.
Get it?
Just a heads up to be aware of all these kinds of things...
The sales Yelp sales reps have access and control of the reviews for each account in their territory.
- by SueYelp March 20, 2009 11:31 PM PDT
- I own a small business in Seattle, very proud of our 12 5 star reviews. But sales calls start coming with each "no thanks" a few of our positive reviews drop off? Why? I see businesses with over 50 reviews but those companies, small like mine are paid advertisers...hum.
- Like this Reply to this comment
-
Showing 1 of 2 pages (26 Comments)So I get a bad review, it was a 2 star rant and it was untrue statments and really inflammatory. I flag it and I get a cookie cuttter response from Yelp that they found the review appropriate and the reviewer is allowed to say how she "feels" about my business. Never mind she refers to another company in her review, so obvious she was a "plant" or a competitor. After my flagging this one 2 start review, others had flagged it as well I get yet another sales pitch. I ask about this suspicious review and I'm told that they have no control over the placement of reviews or bad reviews. I decline their offer. After the most recent decline this ONE bad review out of 12 good reviews was then place at the top of my review page, number one and is now showing up in a google search under my business name on page one of google. What? How does that work? It's outdated, other newer more "useful" reviews are there but THIS is at the top and now showing under my company name when searched.
Now I get yet ONE more call from Yelp and they tell me if I pay 300 dollars a month, while they can't say the review will go away, I can have the power to put my favorite review at the top and the bad review will drop and who knows, perhaps it may just disappear. ARE YOU KIDDING ME! IF THIS IS NOT EXTORTION WHAT IS? Yelp is now, holding me hostage to a false review, won't remove it, won't edit it, ignore ALL the red flags indicating it's a bogus review and are telling me if I just pay up it will all change. Oh, one other little bit of interesting tactic, the refused to put the offer in writing, no email, no letter. She told me they only "deal" over the phone, conversation, nothing in writing.
I think Black Mail applies:
bleed
bribe
extort
payoff
ransom
shakedown
squeeze
threaten
exaction
extortion
hush money
protection
Or Extortion:
1. an act or instance of extorting.
2. Law. the crime of obtaining money or some other thing of value by the abuse of one's office or authority.
3. oppressive or illegal exaction, as of excessive price or interest: the extortions of usurers.
4. anything extorted.
Look at what they do, hunderds if not thousands now of small businesses all with similar stories I am amazed Yelp just does not clean up their act and move on. Investors should be shaking in their boots for sure as they should be.