Apple's mobile-app review system needs overhaul
(Credit:
Apple)
As an iPhone user, one of the things I've found to be increasingly irksome is the customer review system built into Apple's App Store for the iPhones and iPod Touch.
It's as basic as you get, which follows the design ethos found in the many of Apple's hardware products, such as the no-button Mighty Mouse, disappearing MacBook buttons, and I/O ports on its notebook computers and LCD displays.
While simplicity is one of the qualities that makes Apple's products more approachable for the basic user, it's something that doesn't translate well to a crowd-powered review system.
In its current state, the review system lets you very easily rate a software application from one to five stars, along with the option to write in any thoughts or feelings you have about it. This sounds great, in theory, but a good majority of the reviews found on App Store applications seem to prove otherwise.
More often than not, you'll see one-star reviews in which people are raving about the quality of an application. There are also people who give an application five stars, then go on to spend two paragraphs discussing how often it crashes and larger off-topic issues like international pricing and the handset's lack of a copy-and-paste feature. You also get a lot of comments written in ALL CAPS, with lines of Emoji icons, colored stars, and superfluous exclamation marks.
Some sample reviews taken from Tower Bloxx Deluxe 3D FREE, currently the top free title on the App Store.
(Credit: Screenshot by Josh Lowensohn/CNET)In every sense, it's like the Wild West: untamed and full of interesting characters.
To Apple's credit, on Friday, the company (as promised) removed reviews from customers who had not purchased the application they were reviewing. This may cut down on spam and ill-conceived or written reviews, but it's not a big step in improving how the review system works.
Problematic by design
The problem stems from the fact that Apple has treated software reviews with the same level of simplicity it's approached movie and music reviews. These two mediums are not interactive, nor do they have hangups like development schedules and performance issues.
While you can rate an album or music track based on your enjoyment of it, it's not speaking to a truth about frame rate jitters, buggy code, or a developer who has not put out a necessary update in six months--all things you may find in iPhone applications and that can be good to know before plunking down money on a purchase.
One reason there's a lack of these types of clarifications in user reviews is that Apple has fragmented its reviews system based on platform. Mobile users don't get the same quality of review browsing as those using iTunes do. For instance, when viewing user reviews in iTunes, you get the option to flag a bad review and say whether it was helpful. You can also sort by best and worst reviews, along with the most helpful and recent.
On the iPhone, users have none of these options. In fact, there's currently only one way to view reviews--in chronological order. For a device that's slowly gaining independence from having to sync up with a computer (as seen in recent improvements to podcast downloading on the device), this is troubling.
A better system
There are a three things Apple could do, explicitly to software application reviews, that would beef up the system and make reviews really matter to the potential customers who read them. All three can be found on Amazon.com, which has done a really fantastic job of creating a single ratings system that works on multiple genres of products:
1. App Store-iTunes parity
A step in the right direction would be to bring both review systems up to speed with one another. Offer the capability to sort and flag straight from the device. This could be done with sorting buttons at the top of the review section, just like what was done to sort application categories by date and popularity in the latest App Store update.
Amazon's rating system gives you a breakdown of how many votes each star rating received.
(Credit: CNET Networks)Also, let me see a better breakdown of the ratings, like Amazon, Newegg, and others do by showing you how many types of user votes each combined rating is comprised of. Seeing the average rating, which comes out of all the user reviews combined, is helpful. But if I could see that the app has a few hundred four- and five-star ratings, more than the lower numbers, that really says something--especially if I can drill down and read only the reviews with a specific star rating.
2. Developer response
I'd love to see something similar to what Amazon.com allows, in which you can respond to certain reviews with an agreement or rebuttal, something that would let both users and developers approach a concern or praise with feedback of their own.
Using this system, a developer could post direct responses to criticisms instead of the one-star review sitting there, tarnishing an application's score, long after the user had left his or her review. More importantly, the developer would not have to wait on Apple to deal with a review, presuming that it had been flagged by users (who were using iTunes to flag it) before taking action.
3. User credibility
Amazon has something called Real Name. It's a system that lets users back up their reviews by putting their legal name on it. While Apple might frown on the security risks of such a system, it's done great things for Amazon, and has given people who leave detailed and thoughtful reviews a way for others to follow what they're reviewing.
Apply the same thing to the App Store, and you could end up selling more applications from people who are buying things based not on media coverage, or popularity, but purely on the opinions of users whose opinions they respect.
So which one of these things is Apple most likely to add? My guess is the first; a slow trickle-down from iTunes, allowing users to flag bad reviews from the App Store.
Other features from iTunes have slowly crept into the App Store, and in the latest update, Apple seems to have realized that people want more ways to dig through the ever-growing list of applications. I'm just hoping the company will put that same effort into helping us wade through user reviews.
Josh Lowensohn writes for Webware.com, CNET's blog about Web applications and services. E-mail Josh, or follow him on Twitter at http://www.twitter.com/Josh. 





Is there a perfect user review system? Definitely not. Reviews on Amazon have their share of issues as well and it's no different than what's found on iTunes. The part I don't agree with Josh on is allowing devs to rebut, and this has more to do with Apple. I support devs wholeheartedly and get ticked off with people who unfairly write negative reviews on the wrong merits. If an app stinks, then fine, but often, I find reviews just completely offbase. However, if you allow devs to rebut, I just don't think Apple is capable or willing to do this in the right way. Many devs have taken to the blogs and forums asking for user feedback, and I think that's the better way to do this at least until Apple gets its act together.
I think there should (in addition to the purchase/download requirement added) a certain time that a user must use an app before reviewing it. I've found my opinion often changes from the first minute to the next couple of days.
I do like the addition of the prompt of allowing users to rate an app after they delete it. It's a good way to lower the rating of apps that suck, but are still popular simply because it's at the top of a list of something.
Plus, my only 1-star rating was issued by someone who wanted to buy it but thought it wouldn't run on a first-generation iPod Touch, due to my initial carelessness in specifying platform and software requirements.
Beyond that, I would also like to see the improvements mentioned in this article.
In one case, of the the first 3 reviews received, one was with 5 stars, and two of them with 1 star each (from clueless users who thought they were downloading a game and were disappointed when they realized their mistake). The average was set to 2 stars even though the two 1-star reviews were clearly biased. But the worst part is that this 2-star average has not changed after 3 more 5-star reviews were added and one of the 1-star ones was removed (probably because of the offensive words it contained). The average should be instead 4.2 (4 would be accurate enough), not 2. That's less than half the correct number of stars!
My other app shows 3 1/2 but should show 4 1/2 according to the very easy and straightforward manual calculation.
I'm concerned that the lower than actual average may be discouraging users from considering the apps. I'm also frustrated by the apparent lack of any email or feedback page by which one could submit related questions or complaints like these to the App Store.
If the review is scanned for keywords that are positive and keywords that are negative, this could be used to validate the star rating. In other words, a review that uses the words: great, nice, good, etc. should have a star rating of 3 or more. A review containing the words: bad, sucks, awful, etc. should have 3 or less stars. If there is a mix of positive and negative, there wouldn't be any expectation of what the rating should be.
For existing reviews, any that don't seem to make sense could be excluded from the rating calculation. For new reviews, validation would be done immediately and a pop-up could point out the discrepancy to the reviewer and ask for confirmation of the star rating.
Of course, subtlety such as sarcastic remarks would be difficult to read correctly, but this is less likely to be a problem. Most reviews off more obvious praise or criticism.
It was a mostly good idea to remove the ability to review something you had not purchased. There was I observed too many cases where a genuinely good app was simply not understood by people and perhaps because of price it was abused out of existence. The weather section had some great examples of this, pro apps, for pilots etc, being ridiculed.
Something that Josh gets wrong though is developer response. Developers do respond, both in the review system and on their app description page. In fact I have read quite a few fired up developers going into to fight against their (potentially) users. It may not be an 'official this is the developer' channel but it is happening.
I disagree that it isn't working though and considering Josh, as with the rest of the media continues to simply miss the significance of the AppStore, which includes the way the review system is working says to me that the Appstore while far from perfect is ahead of the curve.
- by vbringi08 March 4, 2009 12:57 AM PST
- so if you think about the blackberry, it has much better features, right, and it is good for day traders like myself, don't have one yet since I sit staring at live quotes all day, I do have to disclose i have positons in RIMM and AAPL, but not looking for stock info, but what phone would meet my needs the best?
- Like this Reply to this comment
-
(13 Comments)