Perspective: Misplaced software priorities

We are in danger of losing out in the best and most interesting part of the software market.

I'm referring to the development of high-level components such as user interfaces. These deserve our attention because they increase the value of what we can do with technology. Instead, we're continually re-creating the same low-level infrastructure.

Nobody sees a shared library or a driver, and no one cares about them unless they're broken. Yet that's where developers are focusing much of their time and resources. In doing so, they are making a big mistake--not because these pieces don't deserve resources, but because resources are being thrown at them haphazardly, resulting in waste, not to mention less-than-stellar software. That fact is that much of this software is lacking. Unfortunately, the constant attempts to create and re-create these components generally fall short. And still the resulting products manage to suck up valuable maintenance resources.

We have become resigned to a stagnant desktop experience and too accepting of annoyances such as spam and viruses.

The world of shipbuilding offers valuable lessons on how developers might better proceed. When technologies designed for the bow of a ship mature, they move to the rear of the boat. Even so, they retain their value for long stretches of time. These older technologies usually require less innovation but plenty of renovation.

A different approach prevails in the software world, where resources are continually used to reinvent components such as abstraction layers and other parts of the low-level operating system. These components often are not well built--leaving us with back-end products that are too fragile, and which never become robust, even when they're enhanced later on.

This isn't to say that some rock-solid low-level components have not been done--many core drivers and libraries in Linux and Windows XP, such as mass storage drivers and file systems, are extremely robust. But unfortunately, video drivers, multimedia libraries and recent networking features haven't been produced with the same success. And the quality of printer drivers and subsystems can range from very good to reprehensible.

Unfortunately, many current IT efforts seek to reduce costs by perpetuating low-level software that's less standardized and of low quality. This is a self-destructive and outdated concept that's produced few improvements.

Meanwhile, higher-level functions don't get enough attention. So we have become resigned to a stagnant desktop experience and too accepting of annoyances such as spam and viruses.

Does this mean that we lack the guts to address challenges? I don't believe so. But we need to consolidate our efforts industrywide. It will take decisive leadership and reliance on the open-source development philosophy and tools to help us focus effectively on both innovation and renovation. In doing so, we will be able to concentrate our efforts appropriately, pursuing both design and refinement as warranted by different situations.

The cost of constantly reinventing low-level software (drivers, libraries, hardware abstraction layers and other parts of the low-level operating system) would thus be eliminated. After consolidating our efforts, we'd have room for well-financed and long-overdue robust renovation of the low-level industry-common cores that still need it.

If there's a lesson here, it's to be on guard against lazily lumping together innovation with renovation. By doing so, we get neither a solid engine room nor a fast bow--a mistake no midshipman would ever contemplate.

The big win here would be to kick software innovation into high gear by clearing the decks to focus the innovation segment on the "race to the top" (as Thomas Friedman of The New York Times has put it). People with big dollars then can take big risks for big opportunities.

And in case you don't think it's time yet to confront this imperative, perhaps you should sit back and ponder all the time that you've lost every time your computer jams and you wait anxiously for it to recover. Relax--you've got plenty of time.

Biography
William Jolitz co-wrote 386BSD, an early version of the BSD Unix operating system.

More Perspectives

Too costly

Software manufacturers, to include Microsoft, constantly recycle most of their code, resulting in bloated software and buggy software. It's too costly for them to just renovate part by part when they have a deadline to meet, especially with the market today. Soon, though, our software (US) is going to be so poor, other countries (like China) will surpass us on it. Our poor software engineers will then be just code rewriters.<br /><br />A great example of a downfall because of the misplaced priorities is Netscape. They spun off to Mozilla, tried to use the exisiting netscape engine, failed because it was so terrible, and then finally rewrote it completely for a great product.<br /><br />Engineers should start engineering again and stop rewriting junk. That's what a true engineer does.

Too costly

Software manufacturers, to include Microsoft, constantly recycle most of their code, resulting in bloated software and buggy software. It's too costly for them to just renovate part by part when they have a deadline to meet, especially with the market today. Soon, though, our software (US) is going to be so poor, other countries (like China) will surpass us on it. Our poor software engineers will then be just code rewriters.<br /><br />A great example of a downfall because of the misplaced priorities is Netscape. They spun off to Mozilla, tried to use the exisiting netscape engine, failed because it was so terrible, and then finally rewrote it completely for a great product.<br /><br />Engineers should start engineering again and stop rewriting junk. That's what a true engineer does.

[Richard G.]

Response to author

My issue is with his intentional exlusion of America's Intellectual Property and Copyright laws and practicies. I find it hard to believe someone with his views and experience would not be familar with the way in which IP laws hinder the vision he has for software. We can't just pick ONE kernel, ONE driver interface, ONE sys-call interface, ONE compiler and ONE application binary interface for use on ALL computers. As nice it would be if we could do that, the fact is companies have a vested interest to re-invent the wheel and force customers into THIER OWN Kernel, thier own interfaces, and their own software formats. That's were the money is, hench that's where all our software glut stems from.<br /><br />I take issue with one particular sentance in his article: "Unfortunately, many current IT efforts seek to reduce costs by perpetuating low-level software that's less standardized and of low quality."<br /><br />There's lots of things to niptick here, but let's start with his use of the phrase "less standardized". First of all, what is your standard? What do you consider The Standard? BSD Unix? Java? Also, "many IT efforts ... [chose] less standardized low-level software" could be seen as you critizing the use of Linux in many IT efforts. Linux is non-standard right? And IT projects go to it for low(er) costs.<br /><br />That leads me to the next issue with your sentance, "low quality." This is something that's totally subjective. How do you measure "quality" in software? Few bugs? Good documentation? Good support? Good industry acceptance? Good run-time efficiency? Readable code? Portable Code? All the above? In fact, if we were to talk with Mac users and ask them what "quality software" was all about, they'd likely quip back about pretty colors and cute icons is the only natural way to judge software.<br /><br />Unless you can define a way to measure "quality software" that we can all agree on (because otherwise it wouldn't be Standardized,) and unless you tell us exactly what these magical "Standards" are, none of your article holds ground.<br /><br />Another nitpick I'll take issue with: "So we have become resigned to a stagnant desktop experience and too accepting of annoyances such as spam and viruses."<br /><br />First of all, spam email is a people problem. Bad people generate spam emails, not unstandardized low-quality device drivers. And as for virises, well ... all I can say is that most virises infect their host system through high-level user software (email clients and web browswers.) But then again, things like buffer overruns are a low-level problem and speak to what your article is addressing.<br /><br />Hardware in constantly changing. New products come to market -- sometimes innovative new products -- and they need NEW device driver software written. You can't expect driver software, or any architectural-specific code to just stay the same forever. This software HAS to change, improve, be modified, get ported over time. Compilers have to be updated and improved for new chips and instructions sets, etc. The only way to eliminate the cost of this software would be to ask the hardware industry to stop everything.

[Marcus Westrup]

Not yet a science.

You've touched on what I feel is the real reason for so much confusion in today's software.<br />Software development, despite all the efforts at standardization and streamlining, is still very much an art form. And like art, every artist has his own style.<br /><br />The education system is producing computer science students who are familiar with all the basics, but with a grasp of the "art" that is at the paint-by-number level. They know the steps, but not how to bring it all together. The result is poor code without realizing why.

[bcsaxman]

Somewhat Misplaced Criticism

You correctly pointout the role IP &#38; copyright laws are fowling <br />things up in the US, but I don't think the author is off base with <br />trying to get more resources devoted to interface issues.<br /><br />Your back-hand to the Mac platform is in fact a good example <br />of the value in the author's argument. OSX is, and always has <br />been, more than just pretty icons. It's about as stable as Linux, <br />and is light years ahead of either Linux or Windows in interface <br />design. Both things enhance the user experience <br /><br />For example, back when Mac was in the old Classic days, the <br />ONLY thing that made it desirable were the pretty icons (and <br />maybe the fact that when you, inevitably, had to reboot, you <br />wouldn't encounter a full system lockup, ala Windows). The fact <br />that Apple worked hard on both sides of the issue of usability on <br />the OSX update shows remarkable understanding that you need <br />to devote resources to the whole - not just the back end OR the <br />interface.<br /><br />I think the author is approaching his interface based prescription <br />from the standpoint of the general problems with Linux (bad <br />user interface &#38; good underpinnings) and Windows (mediocre <br />interface and bad underpinnings). Bad to mediocre interface <br />issues are the common problem for 80-90% of the computing <br />world who are using both. So, I believe he's quite right to point <br />out the necessity of addressing it, while also pointing out the <br />benefits of 'under the hood' standardization (regardless of the <br />problems involved in achieving that).

[Richard G.]

Response to author

My issue is with his intentional exlusion of America's Intellectual Property and Copyright laws and practicies. I find it hard to believe someone with his views and experience would not be familar with the way in which IP laws hinder the vision he has for software. We can't just pick ONE kernel, ONE driver interface, ONE sys-call interface, ONE compiler and ONE application binary interface for use on ALL computers. As nice it would be if we could do that, the fact is companies have a vested interest to re-invent the wheel and force customers into THIER OWN Kernel, thier own interfaces, and their own software formats. That's were the money is, hench that's where all our software glut stems from.<br /><br />I take issue with one particular sentance in his article: "Unfortunately, many current IT efforts seek to reduce costs by perpetuating low-level software that's less standardized and of low quality."<br /><br />There's lots of things to niptick here, but let's start with his use of the phrase "less standardized". First of all, what is your standard? What do you consider The Standard? BSD Unix? Java? Also, "many IT efforts ... [chose] less standardized low-level software" could be seen as you critizing the use of Linux in many IT efforts. Linux is non-standard right? And IT projects go to it for low(er) costs.<br /><br />That leads me to the next issue with your sentance, "low quality." This is something that's totally subjective. How do you measure "quality" in software? Few bugs? Good documentation? Good support? Good industry acceptance? Good run-time efficiency? Readable code? Portable Code? All the above? In fact, if we were to talk with Mac users and ask them what "quality software" was all about, they'd likely quip back about pretty colors and cute icons is the only natural way to judge software.<br /><br />Unless you can define a way to measure "quality software" that we can all agree on (because otherwise it wouldn't be Standardized,) and unless you tell us exactly what these magical "Standards" are, none of your article holds ground.<br /><br />Another nitpick I'll take issue with: "So we have become resigned to a stagnant desktop experience and too accepting of annoyances such as spam and viruses."<br /><br />First of all, spam email is a people problem. Bad people generate spam emails, not unstandardized low-quality device drivers. And as for virises, well ... all I can say is that most virises infect their host system through high-level user software (email clients and web browswers.) But then again, things like buffer overruns are a low-level problem and speak to what your article is addressing.<br /><br />Hardware in constantly changing. New products come to market -- sometimes innovative new products -- and they need NEW device driver software written. You can't expect driver software, or any architectural-specific code to just stay the same forever. This software HAS to change, improve, be modified, get ported over time. Compilers have to be updated and improved for new chips and instructions sets, etc. The only way to eliminate the cost of this software would be to ask the hardware industry to stop everything.

[Marcus Westrup]

Not yet a science.

You've touched on what I feel is the real reason for so much confusion in today's software.<br />Software development, despite all the efforts at standardization and streamlining, is still very much an art form. And like art, every artist has his own style.<br /><br />The education system is producing computer science students who are familiar with all the basics, but with a grasp of the "art" that is at the paint-by-number level. They know the steps, but not how to bring it all together. The result is poor code without realizing why.

[bcsaxman]

Somewhat Misplaced Criticism

You correctly pointout the role IP &#38; copyright laws are fowling <br />things up in the US, but I don't think the author is off base with <br />trying to get more resources devoted to interface issues.<br /><br />Your back-hand to the Mac platform is in fact a good example <br />of the value in the author's argument. OSX is, and always has <br />been, more than just pretty icons. It's about as stable as Linux, <br />and is light years ahead of either Linux or Windows in interface <br />design. Both things enhance the user experience <br /><br />For example, back when Mac was in the old Classic days, the <br />ONLY thing that made it desirable were the pretty icons (and <br />maybe the fact that when you, inevitably, had to reboot, you <br />wouldn't encounter a full system lockup, ala Windows). The fact <br />that Apple worked hard on both sides of the issue of usability on <br />the OSX update shows remarkable understanding that you need <br />to devote resources to the whole - not just the back end OR the <br />interface.<br /><br />I think the author is approaching his interface based prescription <br />from the standpoint of the general problems with Linux (bad <br />user interface &#38; good underpinnings) and Windows (mediocre <br />interface and bad underpinnings). Bad to mediocre interface <br />issues are the common problem for 80-90% of the computing <br />world who are using both. So, I believe he's quite right to point <br />out the necessity of addressing it, while also pointing out the <br />benefits of 'under the hood' standardization (regardless of the <br />problems involved in achieving that).

Some serious problems with the examples

I'm not sure what the point of this editorial is. Initially I thought <br />it was going to focus on the importance of user interfaces, but it <br />ended up comparing the stability of disk drivers to that of video <br />drivers as an argument that we shouldn't be focusing on drivers. <br />I'm lost as to the reasoning (although I agree about the <br />importance of user interfaces). But fundamentally, the reason <br />disk drivers are more stable than video drivers is boils down to <br />two basic facts. 1) The video market is changing much faster, so <br />they don't have a chance to settle down. 2) Disk drivers are <br />mission critical, and so are quite reasonably going to undergo <br />more testing.<br /><br />But what does that have to do with user interfaces?

User Interface Designers

To the point of user interfaces: the web browser interfaces are often caught between the QBE (Query By Example) designs of forms that put far too much information on a screen and the web page designed for hyperdistribution. The requirements for a specific, repetitive and high rate job, an occasional user, and the surfer couldn't be more different where productivity is the criteria to be optimized. <br /><br />The lesson that has to be taught to every generation of UI designers is to ask first: who, when, where and why. What will follow. It isn't about putting all of the fields on the page that a theoretical process requires, it is about knowing what the user is actually doing so you can know exactly what they should be seeing.<br /><br />It seems obvious and maybe that is why it is so often neglected. Some companies form committees for this, and even then, the UI dies on the desktop. Some companies have labs that test, observe and actually do work out the motion metrics. They fare quite a bit better if they can focus on a real user and not a class of users.<br /><br />Put away your Jakob Nielsen books and go talk to the people doing the job; not their managers.

Some serious problems with the examples

I'm not sure what the point of this editorial is. Initially I thought <br />it was going to focus on the importance of user interfaces, but it <br />ended up comparing the stability of disk drivers to that of video <br />drivers as an argument that we shouldn't be focusing on drivers. <br />I'm lost as to the reasoning (although I agree about the <br />importance of user interfaces). But fundamentally, the reason <br />disk drivers are more stable than video drivers is boils down to <br />two basic facts. 1) The video market is changing much faster, so <br />they don't have a chance to settle down. 2) Disk drivers are <br />mission critical, and so are quite reasonably going to undergo <br />more testing.<br /><br />But what does that have to do with user interfaces?

User Interface Designers

To the point of user interfaces: the web browser interfaces are often caught between the QBE (Query By Example) designs of forms that put far too much information on a screen and the web page designed for hyperdistribution. The requirements for a specific, repetitive and high rate job, an occasional user, and the surfer couldn't be more different where productivity is the criteria to be optimized. <br /><br />The lesson that has to be taught to every generation of UI designers is to ask first: who, when, where and why. What will follow. It isn't about putting all of the fields on the page that a theoretical process requires, it is about knowing what the user is actually doing so you can know exactly what they should be seeing.<br /><br />It seems obvious and maybe that is why it is so often neglected. Some companies form committees for this, and even then, the UI dies on the desktop. Some companies have labs that test, observe and actually do work out the motion metrics. They fare quite a bit better if they can focus on a real user and not a class of users.<br /><br />Put away your Jakob Nielsen books and go talk to the people doing the job; not their managers.