• On CHOW: Can girls use the guys' bathroom?
February 27, 2009 2:54 PM PST

Apple's mobile-app review system needs overhaul

by Josh Lowensohn
  • Font size
  • Print
  • 13 comments
(Credit: Apple)

As an iPhone user, one of the things I've found to be increasingly irksome is the customer review system built into Apple's App Store for the iPhones and iPod Touch.

It's as basic as you get, which follows the design ethos found in the many of Apple's hardware products, such as the no-button Mighty Mouse, disappearing MacBook buttons, and I/O ports on its notebook computers and LCD displays.

While simplicity is one of the qualities that makes Apple's products more approachable for the basic user, it's something that doesn't translate well to a crowd-powered review system.

In its current state, the review system lets you very easily rate a software application from one to five stars, along with the option to write in any thoughts or feelings you have about it. This sounds great, in theory, but a good majority of the reviews found on App Store applications seem to prove otherwise.

More often than not, you'll see one-star reviews in which people are raving about the quality of an application. There are also people who give an application five stars, then go on to spend two paragraphs discussing how often it crashes and larger off-topic issues like international pricing and the handset's lack of a copy-and-paste feature. You also get a lot of comments written in ALL CAPS, with lines of Emoji icons, colored stars, and superfluous exclamation marks.

Some sample reviews taken from Tower Bloxx Deluxe 3D FREE, currently the top free title on the App Store.

(Credit: Screenshot by Josh Lowensohn/CNET)

In every sense, it's like the Wild West: untamed and full of interesting characters.

To Apple's credit, on Friday, the company (as promised) removed reviews from customers who had not purchased the application they were reviewing. This may cut down on spam and ill-conceived or written reviews, but it's not a big step in improving how the review system works.

Problematic by design
The problem stems from the fact that Apple has treated software reviews with the same level of simplicity it's approached movie and music reviews. These two mediums are not interactive, nor do they have hangups like development schedules and performance issues.

While you can rate an album or music track based on your enjoyment of it, it's not speaking to a truth about frame rate jitters, buggy code, or a developer who has not put out a necessary update in six months--all things you may find in iPhone applications and that can be good to know before plunking down money on a purchase.

One reason there's a lack of these types of clarifications in user reviews is that Apple has fragmented its reviews system based on platform. Mobile users don't get the same quality of review browsing as those using iTunes do. For instance, when viewing user reviews in iTunes, you get the option to flag a bad review and say whether it was helpful. You can also sort by best and worst reviews, along with the most helpful and recent.

On the iPhone, users have none of these options. In fact, there's currently only one way to view reviews--in chronological order. For a device that's slowly gaining independence from having to sync up with a computer (as seen in recent improvements to podcast downloading on the device), this is troubling.

A better system
There are a three things Apple could do, explicitly to software application reviews, that would beef up the system and make reviews really matter to the potential customers who read them. All three can be found on Amazon.com, which has done a really fantastic job of creating a single ratings system that works on multiple genres of products:

1. App Store-iTunes parity
A step in the right direction would be to bring both review systems up to speed with one another. Offer the capability to sort and flag straight from the device. This could be done with sorting buttons at the top of the review section, just like what was done to sort application categories by date and popularity in the latest App Store update.

Amazon's rating system gives you a breakdown of how many votes each star rating received.

(Credit: CNET Networks)

Also, let me see a better breakdown of the ratings, like Amazon, Newegg, and others do by showing you how many types of user votes each combined rating is comprised of. Seeing the average rating, which comes out of all the user reviews combined, is helpful. But if I could see that the app has a few hundred four- and five-star ratings, more than the lower numbers, that really says something--especially if I can drill down and read only the reviews with a specific star rating.

2. Developer response
I'd love to see something similar to what Amazon.com allows, in which you can respond to certain reviews with an agreement or rebuttal, something that would let both users and developers approach a concern or praise with feedback of their own.

Using this system, a developer could post direct responses to criticisms instead of the one-star review sitting there, tarnishing an application's score, long after the user had left his or her review. More importantly, the developer would not have to wait on Apple to deal with a review, presuming that it had been flagged by users (who were using iTunes to flag it) before taking action.

3. User credibility
Amazon has something called Real Name. It's a system that lets users back up their reviews by putting their legal name on it. While Apple might frown on the security risks of such a system, it's done great things for Amazon, and has given people who leave detailed and thoughtful reviews a way for others to follow what they're reviewing.

Apply the same thing to the App Store, and you could end up selling more applications from people who are buying things based not on media coverage, or popularity, but purely on the opinions of users whose opinions they respect.

So which one of these things is Apple most likely to add? My guess is the first; a slow trickle-down from iTunes, allowing users to flag bad reviews from the App Store.

Other features from iTunes have slowly crept into the App Store, and in the latest update, Apple seems to have realized that people want more ways to dig through the ever-growing list of applications. I'm just hoping the company will put that same effort into helping us wade through user reviews.

Josh Lowensohn writes for Webware.com, CNET's blog about Web applications and services. E-mail Josh, or follow him on Twitter at http://www.twitter.com/Josh.
Add a Comment (Log in or register) (13 Comments)
  • prev
  • next
by techgeekdude February 27, 2009 3:36 PM PST
I agree with Josh to a certain degree. I see a lot of reviews where people clearly don't understand the review process...one-star reviews where they say rave about the app is all too common which drags down the overall ratings. There are some who are clearly idiots who give bad ratings simply because the price went down and they feel jilted at having paid a higher price. I think an Amazon-like breakdown would help significantly. The other problem with Apple's review process is that when you delete an app from your device, it automatically asks you to rate it. In my mind, this is completely flawed. Chances are if you're deleting an app from your device, you're probably not satisfied with it so why create that situation in the first place. On top of that, it only allows you to give it a starred rating. You can't even add comments.<br /><br />Is there a perfect user review system? Definitely not. Reviews on Amazon have their share of issues as well and it's no different than what's found on iTunes. The part I don't agree with Josh on is allowing devs to rebut, and this has more to do with Apple. I support devs wholeheartedly and get ticked off with people who unfairly write negative reviews on the wrong merits. If an app stinks, then fine, but often, I find reviews just completely offbase. However, if you allow devs to rebut, I just don't think Apple is capable or willing to do this in the right way. Many devs have taken to the blogs and forums asking for user feedback, and I think that's the better way to do this at least until Apple gets its act together.
Reply to this comment
by Sporlo February 27, 2009 3:50 PM PST
in response to your comment about the rate-after-deletion, maybe other people use it differently than me, but I don't see it as a way to "rape" it. Sure, on average it'd probably lower it's rating, but a lot of the time I'll delete apps simply because I'm not using them often enough, but they are still quality programs.
by Sporlo February 27, 2009 3:46 PM PST
Definitely a good point. I used to pay attention to reviews of apps, but at some point I just got fed up with them and now I base my judgement off the app's description and screenshots (which I find more helpful than the description a lot of the time; I'm always wary of massive advertising of one's own app in the description...).<br /><br />I think there should (in addition to the purchase/download requirement added) a certain time that a user must use an app before reviewing it. I've found my opinion often changes from the first minute to the next couple of days.<br /><br />I do like the addition of the prompt of allowing users to rate an app after they delete it. It's a good way to lower the rating of apps that suck, but are still popular simply because it's at the top of a list of something.
Reply to this comment
by techgeekdude February 27, 2009 4:25 PM PST
I like your idea of creating a "review waiting period". It's easy to get caught up in the hype of a new app, and feelings change as you use an app more. I do know that early reviews can have an impact on whether an app succeeds so that may be an issue. In regards to rating apps upon deleting, I guess we agree to disagree.
by f00br February 27, 2009 4:25 PM PST
The "Developer response " feature also exists at the newegg.com and .ca sites. Certain vendors of products often respond to customers' reviews, especially to comment on negative ones.
Reply to this comment
by billmosby February 27, 2009 5:09 PM PST
First, I'd like to see the average star rating computed correctly. My modest little App, "Crater Naut", currently has 6 reviews totaling 24 stars. Divided by 6, the average should be 4. But it's 3.<br /><br />Plus, my only 1-star rating was issued by someone who wanted to buy it but thought it wouldn't run on a first-generation iPod Touch, due to my initial carelessness in specifying platform and software requirements.<br /><br />Beyond that, I would also like to see the improvements mentioned in this article.
Reply to this comment
by alberto20--2008 March 3, 2009 1:44 AM PST
I have a similar complaint as well. I have two apps and both show an average rating lower than they should. <br /><br />In one case, of the the first 3 reviews received, one was with 5 stars, and two of them with 1 star each (from clueless users who thought they were downloading a game and were disappointed when they realized their mistake). The average was set to 2 stars even though the two 1-star reviews were clearly biased. But the worst part is that this 2-star average has not changed after 3 more 5-star reviews were added and one of the 1-star ones was removed (probably because of the offensive words it contained). The average should be instead 4.2 (4 would be accurate enough), not 2. That's less than half the correct number of stars!<br /><br />My other app shows 3 1/2 but should show 4 1/2 according to the very easy and straightforward manual calculation.<br /><br />I'm concerned that the lower than actual average may be discouraging users from considering the apps. I'm also frustrated by the apparent lack of any email or feedback page by which one could submit related questions or complaints like these to the App Store.
by Jesse Chan February 27, 2009 6:57 PM PST
Sometimes a review is negative for an older version of the app and sentiment changes after an app update or price change. All of a sudden you have positive reviews, but for a possible customer it's hard to know what others' are reviewing against -- app update or price update. It would be useful to have the app version, price, and date next to each user review. This would provide better context to readers to know "what is actually" being reviewed.
Reply to this comment
by AceofThumbs February 27, 2009 9:05 PM PST
I have a suggestion that could clean up old reviews and prevent mistakes in future reviews in the App Store.<br /><br />If the review is scanned for keywords that are positive and keywords that are negative, this could be used to validate the star rating. In other words, a review that uses the words: great, nice, good, etc. should have a star rating of 3 or more. A review containing the words: bad, sucks, awful, etc. should have 3 or less stars. If there is a mix of positive and negative, there wouldn't be any expectation of what the rating should be.<br /><br />For existing reviews, any that don't seem to make sense could be excluded from the rating calculation. For new reviews, validation would be done immediately and a pop-up could point out the discrepancy to the reviewer and ask for confirmation of the star rating.<br /><br />Of course, subtlety such as sarcastic remarks would be difficult to read correctly, but this is less likely to be a problem. Most reviews off more obvious praise or criticism.
Reply to this comment
by krolek222 February 28, 2009 4:21 PM PST
This article is reads like a craigslist rant; well, at least the first few hundred words did. I couldn't make it any further--low tolerance for whining. I mean, really, is this high priority on any person's list these days?
Reply to this comment
by CyStarkman February 28, 2009 7:51 PM PST
The Appstore review system is curious.<br /><br />It was a mostly good idea to remove the ability to review something you had not purchased. There was I observed too many cases where a genuinely good app was simply not understood by people and perhaps because of price it was abused out of existence. The weather section had some great examples of this, pro apps, for pilots etc, being ridiculed.<br /><br />Something that Josh gets wrong though is developer response. Developers do respond, both in the review system and on their app description page. In fact I have read quite a few fired up developers going into to fight against their (potentially) users. It may not be an 'official this is the developer' channel but it is happening.<br /><br />I disagree that it isn't working though and considering Josh, as with the rest of the media continues to simply miss the significance of the AppStore, which includes the way the review system is working says to me that the Appstore while far from perfect is ahead of the curve.
Reply to this comment
by vbringi08 March 1, 2009 11:44 PM PST
I would like to know about the functionality of the iphone vs blackberry. Which has better features for the financial news, I know bberry connects to Etrade Pro, as a feature, does the iphone have a similar link or just browsing, how can i trade on the go is my basic question.? I do acknowledge that I have positions in both RIMM and AAPL, full disclosure is impt.)
Reply to this comment
by vbringi08 March 4, 2009 12:57 AM PST
so if you think about the blackberry, it has much better features, right, and it is good for day traders like myself, don't have one yet since I sit staring at live quotes all day, I do have to disclose i have positons in RIMM and AAPL, but not looking for stock info, but what phone would meet my needs the best?
Reply to this comment
(13 Comments)
  • prev
  • next
advertisement
Click Here

About Webware

Say No to boxed software! The future of applications is online delivery and access. Software is passé. Webware is the new way to get things done.

Add this feed to your online news reader

Webware topics

Google's social side aims for some Buzz

Facebook and Twitter are the darlings of the social-media world, not Google--which hopes to change that with Buzz, betting it can organize your online social life.

Watching the birth of a gaming start-up

Stewart Butterfield and his friends are back at it with a new company. CNET's Daniel Terdiman was given exclusive, behind-the-scenes access as they built it from scratch.

Inside CNET News

Scroll Left Scroll Right