Faster chips are leaving programmers in their dust

Newer computer chips with multiple processors require dauntingly complex software and programmers are having a hard time keeping up.
The New York Times

The story "Faster chips are leaving programmers in their dust" published December 16, 2007 at 9:55 AM is no longer available on CNET News.

Content from The New York Times expires after 7 days.

[Jesse Chan]

Pervasive DataRush

There are a few frameworks to help programmers reach parity with faster, multicore CPUs. One such framework is Pervasive's DataRush: http://fishtrain.com/2007/12/12/interview-with-pervasive-datarush/

[Leria]

Programmers have just gotten lazy

I have to say that I think the problem is just that programmers have gotten lazy.
For nearly 20 years, processors were all single-core. Now, things are changing, and the programmers don't want to change to keep up.
It is not that hard to write software for multi-core, but you have to have that in mind at the beginning and specifically write the software to use the other cores, something that some older programmers don't know how to do or don't want to learn how to do.

[Harlan879]

Not too hard for new software, but try porting MS Office!

It's true that some sorts of applications can be reasonably easily multi-threaded to run on multi-core chips. But it's equally true that it's just about impossible to convert existing software to run efficiently on those chips. Imagine something like Microsoft Office, with millions of lines of code. In order to make that run efficiently on multiple processes you'd practically have to rewrite the application from scratch. That's an application that look multiple thousands of man-years to write. And you'd have to basically do it again, but this time with a more complicated design. It's no wonder the software is lagging the hardware...

[chash360]

They're not lazy, they're outsourced...

The GOV has so undermined our own tech base with things like H1-B Visa's and NAFTA, CAFTA, etc. that most of our programming resources are not state side. Just educating to the current technology does not train people to be creative, the biggest flaw in overseas outsourcing. I firmly believe it is American Culture that breeds and fosters creativity, something most H1-B Visa holders will probably never absorb. I am sure that there are many programmers quite capable of doing this, but none of them want to be sold out by their companies and/or their country anymore, so none step up to the plate. Many have abandonned IT altogether. I know over a dozen languages, assembly on several processors(AMD,HP,Intel,Motorola,Sun,Etc), and believe I myself could solve this problem (and have several proposals) but I don't want to be stuck in the volatile world of IT. I will remain in adjacent fields where I can use my IT skills and hopefully not be outsourced as IT.

[The_Decider]

It is not always possible

To write a program to run in parallel.

Some things simply can not be done in parallel.

Some can.

The trick is understanding when and where parallelism will help, when it is impossible and when it will hurt. That takes more then just programming knowledge, but an understanding of processor architecture.

Yet another reason why all programmers should have a legitimate computer science degree or something close.

[rapier1]

Its not that hard?

Maybe for some classes of problems but for others it can be quite difficult. There is difference between just throwing off a UI thread and actually determining how an algorithm can be properly parallelized for a given architecture. This is assuming that the kind of problem you are trying to solve even lends itself to parallelization. Anyway, since you are a whiz at this sort of thing - do me a favor and come up with an optimal solution for an aes128-ctr implementation that scales, without performance loss from the standard implementation, from 1 to N cores. As note, it needs to run equally well on x86, itanium, xeon, and sunfire archs. As a bonus, a Cell implementation would be swank!

Given then ctr is embarassingly parallel to begin with it should be no problem. Looking foward to your code!

[rdupuy11]

Everyone has it upside down

Programmers don't harness the power of new machines....new machines make it easier to program.

When you had to be efficient, you worried about every bit. Now people will use a 32 bit word as a flag, when you were writing in assembler for the Atari 2600, you sure as heck didn't use more than a single bit for a flag...(it has 128 bytes...thats not a typo, 128 bytes of RAM).

And so it goes, in modern code, you use 32 times the amount of space you need for a flag, because who cares.

You pass around variables, recopying them to functions, even though a pointer would be faster, because who cares.

With multiple cores, you will toss any function or procedure or thread you had in your head to its own core...because who cares.

The hardware makes programming easier. Not harder.

You've all got it backwards. All the power will be used. It won't be used efficiently....it will make programmers more efficient.

It's not the hardware that gets the gold treatment, its the humans.

Get it right. It's like no one is a student of history at all...or they are ashamed to admit the truth of it....but why, its silliness.

[newnewsreader]

Parallel isn't multi-core

Parallel processing is not recent and multi-core processors are no requirement.
Simulation environments (Simula '67), neural nets and OS multitasking are examples that can run on any (single core) hardware. Writing parallel programs is done for 40 years or more with special languages (Occam) or with adapted programming languages, and research is I think almost as old as computing science as a whole.
No easy receipts exist to write a parallel program (as no easy receipts exist to write a 'normal' program for a specific problem), if it is possible at all. And any serious computer scientist should know this.
It is possible to solve the problem, but that would require artificial intelligence on the level of a human programmer.

[OlsonBW]

Problem is the compilers not programmers

Compilers are still horribly manual. THIS is what is keeping
software from advancing as fast as it should.

Imagine having to keep track of every RPM, every spark of a spark
plug, and so on and having to adjust them to keep your car
running.

That in a nutshell is what programmers still have to do with
programs. Compilers are still very much in the Model T stage.
There is --way-- too much that programers have to do that
should have been automated in the 1980s.

Programmers long ago should have stopped needing to do
memory management in their programs. To blame it on "all the
different chips is just plain lame and lazy. There really is just Intel
and AMD. To suggest that companies that write compilers can't
keep track is humorous to me. This is harder that keeping track of
memory management on a computer? Sorry, no.

The same is true with multi-core/multi-processors. As soon as
Intel and AMD started working on this, compiler companies
should have started working on compilers that were aware of this.
They didn't because they didn't need to. They didn't need to
because there isn't enough competition.

[chash360]

Compilers are todays problem, but...

Compilers do need to be vastly improved and updated, but it will take new language constructs and thinking to handle parallel and multicore processing, not just memory management. It will take a much more logical approach to system architecture, than the ad-hoc dos roots of current systems. Detecting Asynchronous from Synchronous instruction stacks, braching and executing appropriately for the number of core resources available, etc. Much of this work was already done at MIT, but industry seems to have such a short memory. I have my doubts they will improve things because again they are probably not addressing security first, which if they did, they would address current compiler issues for single core systems.

[slickuser]

memory management & multi-cores? - STUPID

Your comment is plain stupid! I don't understand what memory management has to do with processors having multiple cores?

Intel provides compilers and tools (thread profiler etc) for optimizing the software for multicores. The author is correct. Companies & its programmers are not able to cope up with processor advancement. They can certainly rewrite their software but doing so would cost them lot of money. So, the best approach would be incremental but it will take time. There are no new products that were developed from scratch coming out. Companies are busy making incremental updates to software that create their main revenue. Most of the new products are web based and they require more than just fast processor.

[Proustian]

You are 100% correct

most modern compilers do their own garbage collecting and memory management, so the programmer doesn't have to anymore. Also Java, C#, and other languages did away with memory pointers and other things that "sloppy code" would cause bottlenecks and slow downs if the programmer didn't meet quality control in his/her code.

Multithreading is only available in most modern languages, and might be the key to writing code that can keep up with modern processors. Some operating systems like Mac OS 9 didn't have multithreading, but Mac OSX fixed that by borrowing it from Unix. Linux has it as well as Windows 2000 and up. OS/2, MS-DOS, CP/M, GEM, Windows 9X, Windows 3.X, etc either don't have multithreading or have an older version that cannot keep up with modern hardware, and also lack multiprocessing abilities. Windows Vista and Mac OSX 10.5 have added in code for the more advanced multithreading and multiprocessing of Dual, Quad, and Octo core processors. Windows NT 4.0 and up had multiprocessing abilities but only in the Workstation and Server versions, but it was limited to a degree.

[The_Decider]

Memory management

Is not only the compilers job. It is almost completely the job of the programmer.

How is a compiler supposed to deal with dynamic allocation? It can not be done.

There exist memory managers for nearly every language because it is not the job of the compiler. In java, does javac care about memory management? No that is the job of the JVM. Does a C++ compiler care about the memory management in the boost libraries? Of course not.

Compiler writers take care of a lot of problems. Why do you think it is that only extremely knowledgeable programmers can write assembly that will beat a good modern C or C++ compiler? Because the compiler takes care of a lot of performance.

The compiler can't do everything. At a certain point it is the programmers job. I realize that so many libraries exist today that programming is no longer a rigorous discipline but more of a paint by numbers affair, but at the end of the day the responsibility lies at the feet of the programmers.

[airbns]

How narrow is your world?

Grove's software spiral didn't start to break down two years ago because Intel chips were generating too much heat, Grove's software spiral and chip strategy changed direction because two years ago, Sun Microsystems announced a new system with eight cores and remarkable energy savings. And by announced, I mean shipped, not some pie-in-the-sky future concept. Everything Intel and AMD have done since then with dual and quad cores has been bandaids on their old strategy in an effort to catch up.

It's remarkable how lame this entire article is. The world does not revolve around Msoft, Intel and AMD. Well maybe it does ;-), but really now, it's amazing that respectable people like the NY Times, source of this article, can talk about them as if all ideas and progress and solutions begin with only them.

More than eight cores in 2010? Try 2008, when Sun comes out with systems that run sixteen. Get with the times, guys. Intel, AMD and Msoft have been scrambling for years now, trying to catch up. Because if Sun one day starts to stretch into the consumer marketplace and Intel hasn't caught up, they'll be in some serious trouble.

It ain't just about Msoft and Intel, guys. Expand your horizons.

[RompStar_420]

I fully Agree

In 1995 Sun came out with the Ultra 1 machine, it had a 64-bit processor, (the machine that I had was running at 167MHz), it was a great piece of equipment and many are still used to this day. The Ultra 2 (which I also owned), had 2 processors, it was a sweet machine running at 200Mhz.

The only problem with Sun is that it is kind of expensive and their Solaris OS is not that easy to use, so thus the fewer users.

If Sun is serious they have to either make the Solaris easy like Apple did with OS X which still (keeping whatever complexities for the users who need and want them).

I havn't seen anything that would indicate that Sun is even serious in attempting to do this with their propiotery Solaris, (which even I don't like, never had), always runned Linux on my Sun equipment, better.

[mattumanu]

Fail transformed into win!

Airbns is a(n) hero.

[Mike_2008]

MultiThreading

I have been writing multi-threaded applications for several years that now can take advantage of the multiple cores in today's processors. The programming model for taking advantage of multiple cores has been available for several years, it's just that some problems are easier to break down and "multi-task" than others. For many complex applications, there are tasks that are not tightly coupled to other tasks that can be given their own thread to run within. For example, a complex game engine could have multiple threads that handle the enemy A.I. - each bot given its own thread (core) that can effectively scale as the number of cores grows. As another example, a web-server could be multi-threaded for handling different connections. More cores would allow the number of simultaneous users to expand. The problem is that the current programming model doesn't make the "threading" of these applications very simple for the average software developer. Also, there are concurrency issues that can arise - deadlocks, etc, that make this type of software more difficult to develop and test. The software that multi-core processors do NOT run faster are the single-threaded applications. It is always amusing to me when someone benchmarks a single-threaded algorithm on a single-core CPU and then compares it to a multi-core CPU. If you want your "algorithm" to scale with multiple cores, it needs to be multi-threaded. How to do this efficiently depends on the algorithm. Some algorithms will never scale to multiple cores because there's not much that can be done in parallel. I would argue that the title of this article should be Faster Chips Are Leaving Some Algorithms In The Dust. There are some of us that can keep all of those cores busy when given the right type of problem.

[dmm]

Limiting factors in parallel computing

There are limiting factors in parallel computing that MUST be kept in mind.

Sequential bottleneck:
If you have a problem that is 50% sequential (i.e., it can NOT be parallelized), then you can NEVER have more than a 2x speed-up in your solution, no matter how many processors you use for the parallelizable part of the problem.

Memory bottleneck:
If (as is typical) your problem requires lots of memory access, then there is no point in using more processors than the memory bus can handle. If you try to get around this by using separate memory for each processor, then you usually run into the problem of keeping the memories synched. You can link the memories with super-high-speed cables, but then that drives costs WAY up. Also, having all those separate memories is a nightmare for an OS.

Storage bottleneck:
Ditto for storage. I've worked on problems where reading the data from hard disk into memory took almost as long as crunching the data. See "sequential bottleneck" above. You can get around this by having each processor use its own hard disk, but again that increases cost, requires additional communication lines, and is a nightmare for the OS.

Bottom line: A parallel computer is not just a regular computer with extra processors. It requires special memory, special memory buses, special memory architecture, special storage, special storage buses, special storage architecture, special interprocessor communication lines, and a specialized OS. This in turn leads to specialized power, cooling, and climate control. Plus there's special programming and special tech support. It is a whole SYSTEM.

[Mike_2008]

Well stated...

From this article, I believe Microsoft is attempting to make parallel computing easily available to the "masses" of developers who are not aware of these issues. I believe this will be terribly difficult to achieve, unless there's a major paradigm shift in the way software is developed.

[Penguinisto]

Good post.

OTOH, there are lots of things happening behind-the-scenes to alleviate most of the bottlenecks. Most multi-core and multi-proc machines do have at least some room and customization built to handle the added CPU horsepower, however.

Sequential bottlenecks can be alleviated somewhat by looking into the sequential pieces and seeing if there isn't some way of approaching the problem from a different angle. Sure, if the sequential parts are going to stay like they are, then you're kind of stuck, and in a lot of cases you have no choice.

Then again, I've seen render engines go from almost pure sequential operation (start calcs and pixel-painting at 0,0 and work your way down to n,n, a given chunk (or "bucket") of pixels at a time) to an almost pure parallelized one (divide the final image/frame into n parts, figure the join results, divide the parts up among separate cores, and use the pre-made join calcs to reconcile the parts for one smooth picture). This is part and parcel of approaching an otherwise sequential job into parallel bits. Sure, you still have the join calcs (depending on engine) to figure down, and optimizing texture maps beforehand has its benefits (subsequent renders are much, much faster) and deficits (that time spent during the first render to optimize the things and stash 'em somewhere in RAM)... but overall, it works. Where once you had to wait for hours (or literally days if you were using a highly complex image or, say, Bryce) to render a 3d image or frame, you can now knock out a reasonable render of a fairly complex one in mere minutes... even with an HT (as opposed to multi-core or SMP) rig.

That said, the bottlenecks have more to do with I/O these days than CPU. The FSB capacity ain't growing at the same rate as the chips that plug into it, and hard disks aren't getting much faster (though internal flash HDD tech may alleviate this for awhile).

This is part of why I picked on Vista elsewhere in the responses... for doing stupid things like using a geek stick for RAM boosting. If you've got a USB keyboard, a USB wireless fob, a USB mouse, a USB connection to your music player, maybe an existing USB external hard disk plugged in... and now you want to use a USB memory stick to speed things up? Heh... if you're lucky, the wad is split among two buses... if you're not, it all shares just one, and that's certainly gonna hurt. Even on a laptop, where it all boils down to maybe just a USB mouse and the geekstick, you're only going to get so much capacity out of that tiny little pipe.

I also picked on Vista because unlike most of its competitors, it won't matter how parallel you make stuff. If you don't optimize for ALL resource usage, you're screwed. Linux has fought for years to keep iowait numbers down to something tolerable wherever possible (and did an excellent job of it IMHO) Hell, look at (ferinstnace) KDE - it was once somewhat of a memory hog. But, each new version is more efficient than the last, reducing memory footprint and bus usage as much as humanly possible... it is now faster than GNOME, and more responsive (again, IMHO) than Aqua on OSX. OSX isn't nearly as efficiency-driven, but since they already know what hardware they're selling it on, they can shoot for optimization on that and call it good... and they keep the pain to a minimum. Vista OTOH went the other way, and for no discernible benefit.

Bleah - methinks that there are a lot of programmers who are not only aware of what's out there, but that actively take advantage of it. Problem is, some vendors don't.

/P

[Penguinisto]

Well, there's one bit of good news in all this.

...someday, Vista will finally have all the computing power that it requires to run at even half the speed of OSX (or even XP) on similar gear. ;)

'course, I wouldn't be holding my breath or anything... :)

/P

[rapier1]

Goodness

With insightful wit like that its a surprise you aren't writing professionally. Let me tell you, whatever you are doing now is *wasting* your vast talent.

[rdupuy11]

You can't tell us what to do

Yes, they should have developed products for machines that hadn't been developed yet, because even though they didn't know any of the spec's, they would eventually show up 20 years later, in one form or another.

C'mon now.

[iBuzz]

Smarter Programmers Needed

It's not that all programmers are lazy (although you can't argue
that some certainly are). It's that smarter programmers are
needed to write code that can do things in parallel on multiple
cores. Perhaps better tools are needed -- ones that can
automatically detect when certain parts of a program can be
carried out in parallel. But until we get to that point, it's like
we're moving back to the days of the 1950's and 1960's where
you actually had to be fairly intelligent in order to program a
computer.

And it leads to blog posts like this one:
http://jbooth.blogspot.com/2007/10/ps3-misconceptions-
and-spin.html

Here, we have a music producer, turned 3D artist, turned game
programmer, who is complaining that the Sony PlayStation 3
cannot run his code fast enough. The problem of course is not
that the PlayStation does not have enough computing power --
it has multiple processors that could easily handle the
computing requirements of a game like Guitar Hero or Rock
Band -- but it's the fact that this guy doesn't know how to use
the parallel processors available to him. He writes all of his
code to be run on a single processor (note his code is geared
towards an architecture that often overheats due to pushing a
single processor to do to much) and claims that the Sony
architecture "sucks" because there is not much in game
programming that can be done in parallel. In actuality, there is
a quite a bit than can be done in parallel, but of course he was
never trained on how to write parallel algorithms (you might
need a college education in computer science to gain that
knowledge!). And he complains that his brain starts to hurt
when he thinks about a solution, which suggests that he might
not have an IQ high enough for the job.

And this gets back to the original point of this article which is
right on. We can't expect script kiddies to write the programs
for these new parallel architectures until we can produce tools
that can do the hard thinking for them.