Gartner: Multicore chips leave software trailing

Gartner sounded a warning on Wednesday about the impact on software of the rapid growth of multicore chips and the number of threads each processor can handle.

In a research note, the analysts argued that software is struggling to keep pace with the fast growth of multicore processors, first from two and four cores per processor, and now to eight and even 32 cores in high-end servers. With 32 processors per socket already shipping, four years from now machines could host 1,024 processors, Gartner said.

Analyst Carl Claunch said: "Many of the software configurations in use today will be challenged to support the hardware configurations possible, and those will be accelerating in the future."

Running advanced multicore machines with today's software is like "putting a Ferrari engine in a go-cart," he said.

Part of the problem, Claunch said, is the speed of innovation in today's chip design. Chips develop with more cores and each core gets more threads as well, adding to the issue, he said. Each generation turns the same number of sockets into twice as many processors.

Claunch said the software that runs today's servers has both hard and soft limits on the number of processors. Part of the problem was that it was difficult to find out what the limits were, he added.

"An operating system might use an eight-bit field to hold the processor number, meaning a hard limit exists of 256 processors," Claunch said. "Soft limits, however, are uncovered only from word-of-mouth, real-world cases. They are caused by the characteristics of the software design, which may deliver poor incremental performance or, in many cases, yield a decrease in useful work as more processors are added."

The issue meant there were limits on the architecture of systems, caused by the software. "The net result will be hurried migrations to new operating systems in a race to help the software keep up with the processing power available on tomorrow's servers," Claunch said.

Colin Barker of ZDNet UK reported from London.

[ppgreat]

Snow Leopard and Windows 7, I think, are going to catch up with all this, right?

[MPB]

Well I know Snow Leopard is, Apples calling it 'Grand Central' but i don't know much about Windows 7.

[pithenumber]

Games that utilize quad cores just comes out, then Intel releases Nehalem with 8 threads.

[Catalina588]

Yup, so now I can be playing a game while encoding a TV show in H.264 without losing half the FPS. More for the same price is better.

[contentcreator--2008]

There's no magic operating-system fix. There are tools to make it easier scattered across the landscape, but bottom-line, the developers need to figure out how to change their apps to take advantage. Rarely is that easy, and sometimes it is downright impossible to parallel-ize individual algorithms. So instead you have to look for parallelism elsewhere in the entire ecosystem, if you're going to take advantage of the cores. That means having parallel-ized display drivers that can process incoming geometry from multiple threads on multiple cores and do parallel processing on it. It'll take a while for the ecosystem to adjust.

Apple's Grand Central for Snow Leopard is an example tool, but won't be able to do anything you can't do already, by definition. There are a variety of other language enhancements too.

[Catalina588]

What's new? Software has been chasing hardware since hardware became "cheap" in the late 1970s. My Windows Vista task manager tells me I have 89 processes and 1348 threads. Don't ask me to go back to a single-core Pentium!

Importantly, newer hardware will greatly ease some big server bottlenecks. I'm thinking of big enterprise Java applications, virtual processing, and encrypting file systems.

Finally, look at benchmarks like TPC-C that have shown 3-4 orders of magnitude improvement over two decades. Competition and better hardware drove those software changes (e.g., relational databases). Any industry performance engineer will agree with me.

[divisionbyzero]

Grand Central in Snow Leopard is a good start. I'm not sure if there is something comparable on Windows 7. In any case it's going to take a lot of refactoring to take advantage of the additional threads/cores.

It's not really the software developers that need to worry however. It's the hardware developers. If program x doesn't run any faster on a newer version of an Intel processor, then why buy a new one? That 's why Intel and Apple (to support its hardware sales) are doing a lot of work to allow software developers to exploit many threads.

[rapier1]

Refactoring is only part of the problem. Not every application can be made serial and even those with large parallel components often have serial bottlenecks that simply can't be routed around. Its going to require some significant shifts in how we look at problems and the assumptions we have. Even then, parallelization is limited by the law of diminishing returns. According to Amdahl's Law even if an application is 95% parallel you'll only see a maximum of a 20x speedup no matter how many cores you throw at it (and it takes around 4096 cores to get there). An application that is 50% parallel will only attain a 100% performance improvement at a maximum. Threading and cores will help but it's going to hit certain, unavoidable, limits dependent on the nature of the application.

[Hey_Radar]

This is why a lot of higher speed dual core CPUs outruns slower quad core CPUs.

[uhpl508]

The problem is usually not the OS but the applications where in some cases they have logic that could be parallelized but is not or possibly can't be because it is an inherently serial process. If you are running a server most of what is running on there is going to probably be good at creating processes to handle parallel tasks and you can use virtualization as well to manage some of the issues. A lot of companies, like Microsoft and Intel and working on ways to make parallization easier and more effective so assuming developers rise to the challenge this will all probably be a non-issue in a few years. Of course, there will always be crappy software and people that don't upgrade. But the OS is not going to parallelize everything for you because it can't.

[jtara]

What a bunch of nonsense, that keeps getting repeated by the technical press year after year after year - for, what, the last 10 years? It's an incorrect, outdated meme.

I'm a software engineer, and used to do a LOT of Windows//MFC work. I can't think of a single project I've worked on in the past several years that didn't rely heavily on multiple threads. Most Windows apps respond well to the additional of multiple cores, as does the OS itself. I've long forgotten about past issues where Windows "froze up" and you couldn't even CTL-ALT-DELETE. It doesn't happen on modern, multiple-core systems.

Gaming is the one category where developers have had to have been dragged kicking and screaming into multi-threading, but I think even that sticking point is long past.

I'm not that familiar with OSX, though I'm a recent convert and learning. From what I now understand, yes, OSX is lighyears behind both Windows and Linux in it's ability to effectively, I've certainly been disappointed with the occasional "spinning ball" with corresponding inability to do anything with ANY program, and inability to, say, stop a large file transfer in Finder once it has started. (This is on the latest Core 2 Duo aluminum MacBook). I look forward to Snow Leopard, and hope it will bring to OSX the stability in this area that Windows and Linux have enjoyed for several years now. (Ducking!)

[jtara]

Oops, sorry, left something out. I meant to write:

"OSX is lighyears behind both Windows and Linux in it's ability to effectively use multiple cores"

[martin1212]

I find it amusing that you start your posting with "what a bunch of nonsense" and then proceed to write - a bunch of nonsense.

Where do you get the idea that OSX is light years behind Windows and Linux? I develop for all three operating systems intensively and OSX does just fine on the threading front. The underlying pthreads implementation is similar to that on Linux, and performs just about (although not quite) as well as the Linux implementation. It is certainly competitive with Windows in terms of performance and functionality.

It sounds like you are actually talking about apps rather than the OS. The OS can provide you a threading API, it is up to developers to use it. If an app is written as a single thread then you will get busy cursors when they are waiting for input, that is just a poorly written app, not the fault of the OS. You get that on Windows too (ever opened a pdf doc in a web browser?)

In terms of systems becoming completely unresponsive, I get that on all three operating systems. I wouldn't say it happens more on OSX than others.

[man_w_balls]

"OSX is lighyears behind both Windows and Linux in it's ability to effectively use multiple cores"

completely wrong, unless you replaced "behind" with "ahead."

Next time you use the Mac, open Activity Monitor and watch the CPU meter tab as you run multiple apps, and you will see that it naturally multitasks multiple tasks, without any need for setting processor affinity.

[jtara]

Regarding the "busy cursor" issue on OSX: I am NOT referring to instances where one gets busy cursor within an app, but are still able to interact with other apps, finder, etc. I've encountered cases where I get a busy cursor and am absolutely unable to do anything until the busy cursor is done. As well (as mentioned in my previous comment) I've encountered the case where I did a large file transfer from my Windows box using Finder, and was unable to interrupt the transfer. These are not "application" problems, but OS problems.

As well, others I know who have Macs have experienced the same problems. One, a student who's probably installed Way Too Much Stuff on his Macbook. The others, professional developers I've worked with who use Macs as their development machines on a job where we were doing Ruby/Rails development, and each had the choice of Windows, Centos, or OSX. A couple chose Windows, I and one other chose Centos (I just think it makes a lot of sense to build on your target platform whenever possible...), and I think we had 4 on Mac Pros. They were constantly banging at the keyboard and cursing at that spinning ball. When I inquired, I was told that "this is normal for Macs". The things people will put up with to use Textmate! ;)

Maybe this is an anomaly. Maybe I've screwed-up my installation somehow. But it's a brand-new machine (bought in December) with very little additional software installed.

My impressions regarding the poor handling of multiple cores by the OS itself are, indeed, not based on a deep understanding of OSX. I pointed-out that I'm pretty-much a complete newbie. That impression is based on Apple's own description of what Snow Leopard will bring to OSX. Apple itself admits that OSX currently just doesn't use multiple cores very effectively.

I didn't criticize Apple's application-threading implementation. Not sure where you read that into my comments. I criticized the OS's inability to effectively/fully isolate apps such that the user still has control over the desktop and other apps when an app goes wacky.

Regarding Activity Monitor. I'd open it when I encounter these situations - IF I COULD.

Don't get me wrong - I love my little MacBook and OSX other than this occasional problem. (And the damn menu at the top of the screen - breaks a pretty fundamental UI principal - put the controls near what they control. It's a LONG WAY UP there on a vertical-orientation screen, and that's the way I run my Macbook on an external display, as this works best for XCode. I quickly figured-out that while Eclipse like a horizontal screen, XCode likes a vertical one. As does TextMate, or, at least, TextMate users. But the menu is a problem when you have, say, a browser window and a couple of other apps open and stacked on top of each other. OK, back in the day nobody flipped their screen sideways, so I can understand why they missed this point. At this point, though, I think this is just a point of stubbornness on the part of Apple...

I do think that Snow Leopard, if it delivers on it's promises, will be a HUGE leap forward for Apple, and will be the biggest incentive for users to continue the trend toward giving Apple greater market share. I usually tend to adopt technologies just before they "go big", and I think OSX is about to do just that, due largely in part to the upcoming "stealth, ho-hum, not much here to see" Snow Leopard.

[MPB]

What do you mean Mac OS X is lighyears behind? Are you CRAZY? After switching from a PC to a Mac the first thing I noticed was how well the software worked with the hardware. I think you need to recall that statement.

[saffroncapital]

In shock news Garnter states blindingly obvious as 'startling new revelation'...

You just wonder what it takes to get adorned with the moniker 'analyst'....

Won it in a Pub Quiz perhaps? Found in a box of breakfast cereal??

[divisionbyzero]

No, you just have to know the right person who already works for that company or one similar to it. Analysts and tech. bloggers tend to be at least a month behind everyone who knows what's really going on and even then they need someone who knows that they are doing to spoon fed it to them.

[NocturnalCT]

Multi threaded programming will seperate the men and women from the boys and girls in software development. At some point monolithic 8 year old VB6 code won't compete against more modern MT code because people that traded their 3+ GHz P4 for a 2.3 Quad Core will expect better performance and won't get it.

This is true in all kinds of environments I'm sure but I see it in astronomical image processing. There are plenty of legacy applications that can't keep up with increased image sizes and numbers. You wait and wait for things to complete. So I use software that *is* multi core aware. Eventually the other guys will come around and add multi threading where possible. I've already seen in happen with more free thinking developers.

Waiting for Nehalem based computers to come down in price just a tad and then I'm picking one up :)

[inachu]

This is good news.
Once an OS properly handles a multi core cpu then the pc slows down.
I am so tired of seeing my new pc bogged down by a newer OS

[drsrca]

The answer is obvious--share the wealth. The average user only needs a tiny fraction of today's dual core. At CES, ncomputing showed 30 typical office workers sharing a $1000 quad core box. Imagine how many users could share 8, 16 , 32 or more cores.

[Imalittleteapot]

The complaint basically boils down to this. Why isn't software getting bloated enough to keep up with multicore chips? It's simple. Because people can't handle anymore bloat. Lately software has been going the other direction. Lighter software that doesn't do much. That's because programs like Office or Windows or even your media player already have more options than you'll ever find in a lifetime. Bring up the options dialog in any program you use and see if you can tell me what all those options do. No, well that's just the friggin options dialog!

We've reached a point where we're not going to be able to bloat software anymore without finding a new interface paradigm that allows the average person to harness that power effectively. This is why everybody is working on multi-touch right now. Don't know if that'll work, but that's why. The big shots already know this, that we need a new paradigm.

Now, I don't know what the next killer app is. Whatever comes along whether it be multi-touch or whatever will eventually find a way to use those cores. The problem is you have to have the hardware before you code for it. Just give it a little more time to catch up.

However, we have to ask a question here. Let's say I have two programs running and two cores. Program A is running on core 1 and program B is running on core 2. Now we rewrite both programs so half their work is distributed among the two cores. Now program A uses half the power of core 1 and core 2 an program B uses half the power of core 1 and core 2. Did I really accomplish anything?

All I've done is make the programs harder to debug. Maybe we should focus on running more single thread processes more effectively where the operating system can intelligently handle the situation instead of letting every program on the computer just deadlock all the cores. If we do that we won't gain a thing. Splitting all the programs up into threads basically leaves me back in single core world because now I don't have any free cores to run any other programs in the background do I? Just let program A run on core 1. It might go slow, but I've got core 2 that I can still use to get my work done while program A does its thing in the background. If you "Super Thread" it just locks me out of all my cores and now I'm still waiting on my computer to finish when I could be getting other work done.

[H4MM3R]

I want my next Laptop to have an Intel i7 ish processor. Sorry I want more cores and more threads with higher CPU speed, that includes the on die bus.

[Endbringer]

People who argue that is is a non-issue as it's always been the case are missing the point. The point is that the software makers are going to have to change and adapt to multi-cores. In the past that was not the case, they just added new features and they worked faster on the newer chips. But now, they have to rewrite most of their software to become multi-threaded. This is a big difference than what happened in the past. I'm a civil designer and you'd think AutoCAD would take advantage of multi-cores, but it does not do so effectively because the core of AutoCAD hasn't changed with the advent of cheap, multi-core processors.

But the real question should be, can a word processor or a spreadsheet program really run faster if it's mutli-threaded? Those two software are most likely the most used in the world. Games, databases, CAD programs, imaging programs, and other 3D programs seem to be about the only software that can actually gain a noticeable advantage of being multi-thread.

In the end the software makers have to change the way they work to take advantage of the new processors, which they are doing really slowly.

[ilya_cilk]

Multicore is here ? dual cores have been out for a couple years now, and quad-cores are fairly common. In fact, it?s hard to find a single-core desktop machine on any vendor site. This scenario is playing out in a similar manner in the data center, where increasingly ?fatter? nodes are featuring more and more cores. Indeed, just a couple months ago, Intel announced a 6-core Xeon processor.

So, multicore is here, yet the vast majority of applications have not been multicore-enabled. Why? Well, here?s the rough calculus done today by software development teams: On a dual-core system (the typical configuration today), one can typically expect a two-fold improvement at best. This performance improvement is just not compelling enough to make the (potentially considerable) development investment. Once the 8- and 16-core systems become mainstream, however, a single-threaded application may be leaving an order of magnitude in performance and throughput on the table. That?s when multicore enablement moves from ?nice to have? to a ?must have.?

Here's an e-Book on multicore programming: http://www.cilk.com/multicore-e-book/