Version: 2008

July 7, 2006 4:00 AM PDT

Vista's PC-rating tool gets a revamp

  • 47 comments

(continued from previous page)

Microsoft has also clarified its intent for the program and offered up more details on how it will work.

"Windows Experience Index is designed to help average consumers easily understand their Windows Vista PCs overall performance, and to simplify the process of determining whether certain software applications will run smoothly based on their system components," it said.

special coverage
Piecing together Vista
Read all the latest on the upcoming Windows update.

It added that computer makers and retailers can use Windows Experience Index to "help customers understand the general performance capabilities of a Windows Vista PC and the types of experiences it can support, relative to other Windows Vista PCs they offer, thereby enabling easy-to-understand differentiation."

Software makers will also benefit, as they can use the scores as to communicate the minimum and recommended system performance for their programs, Microsoft said.

Scores on the doors
The initial version of the tool will offer a 1-to-5 ranking, but additional numbers will be added over time, Microsoft said. That means a PC that gets a "3" today will continue to get a "3", but eventually there will be computers that rank a "6" or a "7".

"We will introduce new scores (6.0-6.9, 7.0-7.9, and so forth) periodically as new hardware component performance capabilities increase," Microsoft said in a statement. "The existing numbers will simply remain defined 'as-is' (i.e. the performance metrics for a 3.2 will always be the same regardless of how many new numbers are added). We created this scoring system with upward scalability and hardware technology advances in mind."

The company also offered some assessment of what the ratings mean, though it did not provide a complete breakdown of each score and what it signifies.

"Using this base score, a customer can then determine if their machine will support their desired experiences (i.e., if a person uses their Windows Vista PC for office productivity and Web browsing, a base score of 1-2 would suit their purposes; or if another person uses their Windows Vista PC for HDTV recording and 3D gaming, a base score of 4-5 would suit their purposes)," Microsoft said.

Dodd said ATI's main concern is that the score will represent an accurate reflection of what the graphics chipmaker's components can do in real-world use.

"I know they are still making changes to what determines whether you get a '4' or a '5'," he said. "As long as Microsoft makes it clear what each rating means and why they are getting that rating, it is a very good thing for end-users."

AMD's Simmons said that even if the tool underestimates the importance of a PC's processor, it is still a benefit if it leads to fewer returned computers and a boost in customer satisfaction. Although it will ultimately be up to PC makers and retailers how to use the scores generated by the Windows Experience Index, Simmons said she envisions the ratings appearing on the kinds of fact sheets that accompany a PC on store shelves.

"Even if it is not necessarily good for the CPU itself, it is good for the industry," Simmons said.

Previous page
Page 1 | 2

See more CNET content tagged:
assessment, chip company, overall, Microsoft Windows Vista, component

Add a Comment (Log in or register) Showing 1 of 2 pages (47 Comments)
An evolving benchmark?
by Christopher Hall July 7, 2006 5:36 AM PDT
They're well-intentioned, no doubt, but I just don't think it's for me, as is likely the case for most people who frequent this site. I can see the Average Joe using it, though, since he's not usually acquainted with the nuts and bolts of his brand new Best Buy computer.

I don't think people who can rattle off their hardware specs from memory, though, exactly fit into the target demographic. (Nor are we supposed to, I'd gather.)
Reply to this comment
An evolving benchmark?
by Christopher Hall July 7, 2006 5:36 AM PDT
They're well-intentioned, no doubt, but I just don't think it's for me, as is likely the case for most people who frequent this site. I can see the Average Joe using it, though, since he's not usually acquainted with the nuts and bolts of his brand new Best Buy computer.

I don't think people who can rattle off their hardware specs from memory, though, exactly fit into the target demographic. (Nor are we supposed to, I'd gather.)
Reply to this comment
Gaming the system
by Neo Con July 7, 2006 7:06 AM PDT
I can see one issue with this could be that hardware manufacturers start building their chips to "game" the rating system instead of concentrating on actual performance. This has been seen time and time again, from Deep Blue being programmed to defeat only one specific chess master to nVidia & ATI building in instructions that give them a higher benchmark score without really being much faster overall.

It would almost be better for Microsoft to not let on what their exact calculations are, but even if they did that (not likely, considering the likely outcry), reverse-engineering would still necessitate that they maintain constant vigilance in the face of potential abuse of the rating system by hardware manufacturers.
Reply to this comment
No kidding...
by Penguinisto July 7, 2006 1:50 PM PDT
You wouldn't be far off from the mark, and this wouldn't be the first time, either:

There was once a HUGE stink over a certain video card manufacturer who tweaked their Quake2 benchmarks (back when certain games and their FPS ratings were the definitive benchmarks) by having the chipset automatically override and downgrade certain barely-visible video options of the game. One of the hardware reviewers caught it (noticed the downgraded mipmapping, I think), renamed "quake2.exe" to "quack2.exe", ran the test again, and found the FPS numbers were far below the published benchmarks.

I can easily see certain makers (not just the CPU-making ones, either) tweaking firmware just to bounce the benchmarks in their favor...

It's bad enough that Vista is going to require some serious horsepower just to operate at anything near the user-perceived speed of its predecessors (...and for what, exactly? A knock-off Aqua clone at the UI and some half-patched attempts at security?) It's even worse given Microsoft's historical habit of seriously underestimating minimum requirements. But now... you get to contend with skewed benchmarks from manufacturers to boot.

Man, I'll readily admit that I'm not a fan of Windows (and rarely if ever have to use it @ work -- never at home)... but damn it's gonna be a rough and expensive ride for those who do use the thing on a regular basis.

/P
Gaming the system
by Neo Con July 7, 2006 7:06 AM PDT
I can see one issue with this could be that hardware manufacturers start building their chips to "game" the rating system instead of concentrating on actual performance. This has been seen time and time again, from Deep Blue being programmed to defeat only one specific chess master to nVidia & ATI building in instructions that give them a higher benchmark score without really being much faster overall.

It would almost be better for Microsoft to not let on what their exact calculations are, but even if they did that (not likely, considering the likely outcry), reverse-engineering would still necessitate that they maintain constant vigilance in the face of potential abuse of the rating system by hardware manufacturers.
Reply to this comment
No kidding...
by Penguinisto July 7, 2006 1:50 PM PDT
You wouldn't be far off from the mark, and this wouldn't be the first time, either:

There was once a HUGE stink over a certain video card manufacturer who tweaked their Quake2 benchmarks (back when certain games and their FPS ratings were the definitive benchmarks) by having the chipset automatically override and downgrade certain barely-visible video options of the game. One of the hardware reviewers caught it (noticed the downgraded mipmapping, I think), renamed "quake2.exe" to "quack2.exe", ran the test again, and found the FPS numbers were far below the published benchmarks.

I can easily see certain makers (not just the CPU-making ones, either) tweaking firmware just to bounce the benchmarks in their favor...

It's bad enough that Vista is going to require some serious horsepower just to operate at anything near the user-perceived speed of its predecessors (...and for what, exactly? A knock-off Aqua clone at the UI and some half-patched attempts at security?) It's even worse given Microsoft's historical habit of seriously underestimating minimum requirements. But now... you get to contend with skewed benchmarks from manufacturers to boot.

Man, I'll readily admit that I'm not a fan of Windows (and rarely if ever have to use it @ work -- never at home)... but damn it's gonna be a rough and expensive ride for those who do use the thing on a regular basis.

/P
Cool
by lovejessii July 7, 2006 8:45 AM PDT
I didn't really understand that very much. But what I did sounded like Microsoft is making some big changes that will improve people's lives. Sounds like Microsoft is going to be doing a lot of good for people. I think all of the technicians should be recognized by Gates and be celebrated as heros.
Reply to this comment
wait till you've been 'round a while longer
by jabbotts July 7, 2006 9:49 AM PDT
your enthusiasm is refreshing and likely not the result of watching DOS evolve into winXP.

Unfortunately, the changes discussed here are only changes to the system measurement tool they've released for us lowly users to confirm how much new hardware we'll have to buy to support Vista.

Thus far, Microsoft's main concern has been doing a lot of good for Microsoft while giving lip service to the good of the user. Perhaps Vista will break the mold but more likely, it'll be another layer of programming code ontop of the all the other layers they've added to the onion called Windows since putting the first layer over DOS.

Like IE7, you can be sure that Vista will be pretty with all the rough edges smoothed over and polished. You can be sure that the hardware required to push the operating system alone will be huge jump from what your on now. You can be sure that there will be complicated license validation schemes in place. You can be sure that Vista will become quite usable by SP1 or SP2 when they get most of the whole plugged. But I base this all on having grownup with the history of the company.

I'm not going to tell you to buy a Mac or download Linux distributions. If you like Vista when it comes out, then use it; just please, read enough to know that they are not the only OS or Application provider in town.

Who knows, perhaps with Bill gone and the desk asignments shuffled around, M$ will prove us Tech folks wrong and you user folks right.
Classic
by qwerty75 July 9, 2006 1:39 AM PDT
At least you openly admit that you don't understand this, which makes sense for a MS fan.

Vista is a poorly implemented knockoff of a 4-5 year old operating system(OSX) that requires way too much hardware just to run it. You have a computer that can barely handle Vista? Congrats, you won't be able to actually run anything on it, without upgrading hardware.

This in no way will improve anyones lives. Vista is already a failure and it hasn't been released. How can say that? Well, for starters the best features have been removed because they couldn't get things to work right, what other OS's have had for years and even decades. It is built on a unsecure platform, and most of the announced security features are extremely weak and amatuerish. There is simply nothing in Vista that someone else hasn't done already, and better.

Yeah, Gates is a hero if you value lying, stealing, illegal and unethical business practices.

Thank you for reinforcing my thoery that MS fans are ignorant about computers.
View all 3 replies
Cool
by lovejessii July 7, 2006 8:45 AM PDT
I didn't really understand that very much. But what I did sounded like Microsoft is making some big changes that will improve people's lives. Sounds like Microsoft is going to be doing a lot of good for people. I think all of the technicians should be recognized by Gates and be celebrated as heros.
Reply to this comment
wait till you've been 'round a while longer
by jabbotts July 7, 2006 9:49 AM PDT
your enthusiasm is refreshing and likely not the result of watching DOS evolve into winXP.

Unfortunately, the changes discussed here are only changes to the system measurement tool they've released for us lowly users to confirm how much new hardware we'll have to buy to support Vista.

Thus far, Microsoft's main concern has been doing a lot of good for Microsoft while giving lip service to the good of the user. Perhaps Vista will break the mold but more likely, it'll be another layer of programming code ontop of the all the other layers they've added to the onion called Windows since putting the first layer over DOS.

Like IE7, you can be sure that Vista will be pretty with all the rough edges smoothed over and polished. You can be sure that the hardware required to push the operating system alone will be huge jump from what your on now. You can be sure that there will be complicated license validation schemes in place. You can be sure that Vista will become quite usable by SP1 or SP2 when they get most of the whole plugged. But I base this all on having grownup with the history of the company.

I'm not going to tell you to buy a Mac or download Linux distributions. If you like Vista when it comes out, then use it; just please, read enough to know that they are not the only OS or Application provider in town.

Who knows, perhaps with Bill gone and the desk asignments shuffled around, M$ will prove us Tech folks wrong and you user folks right.
Classic
by qwerty75 July 9, 2006 1:39 AM PDT
At least you openly admit that you don't understand this, which makes sense for a MS fan.

Vista is a poorly implemented knockoff of a 4-5 year old operating system(OSX) that requires way too much hardware just to run it. You have a computer that can barely handle Vista? Congrats, you won't be able to actually run anything on it, without upgrading hardware.

This in no way will improve anyones lives. Vista is already a failure and it hasn't been released. How can say that? Well, for starters the best features have been removed because they couldn't get things to work right, what other OS's have had for years and even decades. It is built on a unsecure platform, and most of the announced security features are extremely weak and amatuerish. There is simply nothing in Vista that someone else hasn't done already, and better.

Yeah, Gates is a hero if you value lying, stealing, illegal and unethical business practices.

Thank you for reinforcing my thoery that MS fans are ignorant about computers.
View all 3 replies
How lame....
by SystemsJunky July 7, 2006 9:36 AM PDT
What dont they understand. Im running Vista B2 on a MacBook Pro which scored a 3..My pc at home Scored a 5.

It rated My Macbook like so:
Processor: 5
Memory: 4.5
HDD: 3.5
Graphics: 3.8
Gaming Graphics memory : 4.1

My PC Scored 4-5 on everything. There is nothing to the performance rating system that I dont understand. Hell, It even told me which drivers were causing windows to start slowly, shut down slowly, and which programs or drivers that were interfering with sleep mode..Id say its a pretty damn good tool, even in its beta stage..

Are people getting dumber'er? Or is it just me.

Besides that, The help documentation clearly states what the tool is for: Stating "The Windows System Performance Rating measures the capability of your computer's hardware configuration and expresses this measurement as a whole number. A higher performance rating means your computer will perform better and faster, especially when performing more advanced and resource-intensive tasks, than a computer with a lower performance rating. You can use the performance rating number to confidently buy programs and other software that are rated to match your computer's performance level. For example, if this computer has a System Performance Rating of 3, then you can confidently purchase any software designed for this version of Windows that requires a computer with a rating of 3 or less. Etc...."

Im not sure what everyone is complaining about..It makes plenty of sense to me and the ppl i've shown..My mom even understands it...So whats the damn problem?
Reply to this comment
How lame....
by SystemsJunky July 7, 2006 9:36 AM PDT
What dont they understand. Im running Vista B2 on a MacBook Pro which scored a 3..My pc at home Scored a 5.

It rated My Macbook like so:
Processor: 5
Memory: 4.5
HDD: 3.5
Graphics: 3.8
Gaming Graphics memory : 4.1

My PC Scored 4-5 on everything. There is nothing to the performance rating system that I dont understand. Hell, It even told me which drivers were causing windows to start slowly, shut down slowly, and which programs or drivers that were interfering with sleep mode..Id say its a pretty damn good tool, even in its beta stage..

Are people getting dumber'er? Or is it just me.

Besides that, The help documentation clearly states what the tool is for: Stating "The Windows System Performance Rating measures the capability of your computer's hardware configuration and expresses this measurement as a whole number. A higher performance rating means your computer will perform better and faster, especially when performing more advanced and resource-intensive tasks, than a computer with a lower performance rating. You can use the performance rating number to confidently buy programs and other software that are rated to match your computer's performance level. For example, if this computer has a System Performance Rating of 3, then you can confidently purchase any software designed for this version of Windows that requires a computer with a rating of 3 or less. Etc...."

Im not sure what everyone is complaining about..It makes plenty of sense to me and the ppl i've shown..My mom even understands it...So whats the damn problem?
Reply to this comment
I don't need it
by paulsecic July 7, 2006 10:14 AM PDT
because new computers with vista will cost more. Therefore I plan to buy a Mac.
Reply to this comment
In the interest of fairness...
by Penguinisto July 7, 2006 2:00 PM PDT
New macs ain't cheap, either (unless you want a Mini, which IMHO looks and feels severely crippled next to my dual G5 PowerMac).

Vista will certainly require hopped-up hardware, and cheap Dells with a Celeron, 256MB of RAM and a cheap GeForce2 clone ain't going to cut it this go 'round, either. OTOH, I suspect that Dell et. al. will find a way to lash together a cheap enough box for ~ $750 or so (initially) that will just barely cut the mustard.
View reply
Why?
by Andrew J Glina July 8, 2006 8:46 PM PDT
It runs on my Celeron 1.7 256 MB that I bought four years ago, and pre-release code is always bloated with de-bug and test features. Thus any computer bought in the last 2 years should be OK.
I don't need it
by paulsecic July 7, 2006 10:14 AM PDT
because new computers with vista will cost more. Therefore I plan to buy a Mac.
Reply to this comment
In the interest of fairness...
by Penguinisto July 7, 2006 2:00 PM PDT
New macs ain't cheap, either (unless you want a Mini, which IMHO looks and feels severely crippled next to my dual G5 PowerMac).

Vista will certainly require hopped-up hardware, and cheap Dells with a Celeron, 256MB of RAM and a cheap GeForce2 clone ain't going to cut it this go 'round, either. OTOH, I suspect that Dell et. al. will find a way to lash together a cheap enough box for ~ $750 or so (initially) that will just barely cut the mustard.
View reply
Why?
by Andrew J Glina July 8, 2006 8:46 PM PDT
It runs on my Celeron 1.7 256 MB that I bought four years ago, and pre-release code is always bloated with de-bug and test features. Thus any computer bought in the last 2 years should be OK.
Of course Intel doesn't like it...
by chassoto--2008 July 7, 2006 10:24 AM PDT
Things like "better graphics performance" can be had without
requiring the purchase of an entirely new system. "More cores" is
exactly what Intel wants, because it forces yet another
(unnecessary) system swap that Intel benefits greatly from.
Reply to this comment
Of course Intel doesn't like it...
by chassoto--2008 July 7, 2006 10:24 AM PDT
Things like "better graphics performance" can be had without
requiring the purchase of an entirely new system. "More cores" is
exactly what Intel wants, because it forces yet another
(unnecessary) system swap that Intel benefits greatly from.
Reply to this comment
Is this an open door for synthetic benchmark abuse?
by MattLPMP July 7, 2006 8:38 PM PDT
As was the case 5 years ago when the video card wars came to a head, are we going to see the return of specifically-tweaked drivers aimed at improving these "Vista" scores?

Why would they bother with broad hardware offerings when they can govern a driver to enforce market segmentation?

This really doesn't bode well unless Microsoft also randomizes the test.
Reply to this comment
Is this an open door for synthetic benchmark abuse?
by MattLPMP July 7, 2006 8:38 PM PDT
As was the case 5 years ago when the video card wars came to a head, are we going to see the return of specifically-tweaked drivers aimed at improving these "Vista" scores?

Why would they bother with broad hardware offerings when they can govern a driver to enforce market segmentation?

This really doesn't bode well unless Microsoft also randomizes the test.
Reply to this comment
This should be open sourced
by rklrkl July 8, 2006 4:09 AM PDT
I'm sorry, but you just can't trust this rating if the source code isn't available for public inspection. Who's to say that Nvidia, say, haven't paid Microsoft to add 0.5 to the rating if it's an Nvidia card, regardless of how good that card is? It's akin to the driver tweaking that ATI/Nvidia are often accused of to boost their scores in 3D graphic benchmarks.

The actual idea of this isn't a bad one though (the way MS have implemented it is, of course, absolutely awful - it should be an average, not a base score). Maybe it's something some Linux distros should take up (i.e. release a open source Linux, Mac OS X and Windows score suite to analyse how suitable the hardware is to run Linux - including any 3D desktop stuff that's now the rage)? It'll be eye-opening for users how integrated 3D cards provide as good as (if not better) 3D desktop effects on Linux compared to the higher-powered separate 3D cards needed to run Vista's Aero.
Reply to this comment
This should be open sourced
by rklrkl July 8, 2006 4:09 AM PDT
I'm sorry, but you just can't trust this rating if the source code isn't available for public inspection. Who's to say that Nvidia, say, haven't paid Microsoft to add 0.5 to the rating if it's an Nvidia card, regardless of how good that card is? It's akin to the driver tweaking that ATI/Nvidia are often accused of to boost their scores in 3D graphic benchmarks.

The actual idea of this isn't a bad one though (the way MS have implemented it is, of course, absolutely awful - it should be an average, not a base score). Maybe it's something some Linux distros should take up (i.e. release a open source Linux, Mac OS X and Windows score suite to analyse how suitable the hardware is to run Linux - including any 3D desktop stuff that's now the rage)? It'll be eye-opening for users how integrated 3D cards provide as good as (if not better) 3D desktop effects on Linux compared to the higher-powered separate 3D cards needed to run Vista's Aero.
Reply to this comment
NONE of my 3 PC's will handle Vista...:-(
by gary85739 July 9, 2006 7:00 PM PDT
but...since APPLE will release Leopard about the same time that MSFT releases Vista, well...:-)

iMac here I come!

ps: with my NEW iMac I'll run Leopard & Linux, so, goodbye MicroSoft!
Reply to this comment
Have you tried?
by Andrew J Glina July 9, 2006 11:03 PM PDT
I have a four year old Celeron 1.7 with 256 MB that runs Vista. Anything that is under six years should be able to run Vista. Anything that was bought two years ago should run it good. Only if you want all of the flashy features then you will need a new computer. What do you have, PIIs?
NONE of my 3 PC's will handle Vista...:-(
by gary85739 July 9, 2006 7:00 PM PDT
but...since APPLE will release Leopard about the same time that MSFT releases Vista, well...:-)

iMac here I come!

ps: with my NEW iMac I'll run Leopard & Linux, so, goodbye MicroSoft!
Reply to this comment
Have you tried?
by Andrew J Glina July 9, 2006 11:03 PM PDT
I have a four year old Celeron 1.7 with 256 MB that runs Vista. Anything that is under six years should be able to run Vista. Anything that was bought two years ago should run it good. Only if you want all of the flashy features then you will need a new computer. What do you have, PIIs?
Vista Vista Vista
by aqvarivs July 10, 2006 1:25 PM PDT
I'm not anti MS, but am I the only one who doesn't see too much new in the OS other than having become more system needy. No one has said what the operational differences really are that would intice anyone away from XP. I'd really like to know how they differ outside of Vista needing 4GB processors and 4GB of memory and a wopping big Video card and all the latest hardware. Buying a $3,000.00 dollar computer to run a $500.00 dollar OS certainly does not endear me to Vista.
Reply to this comment
Vista Vista Vista
by aqvarivs July 10, 2006 1:25 PM PDT
I'm not anti MS, but am I the only one who doesn't see too much new in the OS other than having become more system needy. No one has said what the operational differences really are that would intice anyone away from XP. I'd really like to know how they differ outside of Vista needing 4GB processors and 4GB of memory and a wopping big Video card and all the latest hardware. Buying a $3,000.00 dollar computer to run a $500.00 dollar OS certainly does not endear me to Vista.
Reply to this comment
Showing 1 of 2 pages (47 Comments)
advertisement

Latest tech news headlines

RSS Feeds

Add headlines from CNET News to your homepage or feedreader.

More feeds available in our RSS feed index.

Markets

Market news, charts, SEC filings, and more

Related quotes

Microsoft (0.00%) 0.00 30.48
Dow Jones Industrials (0.00%) 0.00 10,428.05
S&P 500 (0.00%) 0.00 1,115.10
NASDAQ (0.00%) 0.00 2,269.15
CNET TECH (0.00%) 0.00 1,646.41
  Symbol Lookup
advertisement

Inside CNET News

Scroll Left Scroll Right