How to blow $3 million in taxpayer funds (UPDATED)

You read things like this and it makes you wonder why IT departments continue to throw away money on proprietary licenses why you bother paying taxes at all [See note below]. In this case, the Public Company Accounting Oversight Board (PCAOB) spent $3 million on a document management system from Documentum, and it has been a complete waste of money.

The PCAOB tries to blame itself for the waste, but the reality is that it's a combination of overpriced and overly complex software wrapped in a proprietary license, and an effort to force-feed square-peg technology onto round-peg people.

Open source could have dramatically reduced the PCAOB's risk:

The PCAOB has not sufficiently defined its business and technology requirements for document management. Evaluation of options for document management began in 2003. However, as of September 2006, Documentum (the selected technology), was only being used in a limited capacity. This occurred, in part, because the PCAOB did not fully implement certain best practices for document management projects. As a result, the PCAOB has spent about $3 million on a document management system that is not fully utilized, and the benefits of a robust records and information management program are not being gained.

While much effort by the PCAOB's Office of Information Technology (OIT) has been put into documenting requirements for enterprise and inspections versions of a document management system, the majority of PCAOB employees are still not using the system, and the project lacks overall strategic direction. Some potential users of the system do not view the document management system as fulfilling their business needs.

Unfortunately, I'm guessing this scenario feels painfully familiar to many IT people reading this. As mentioned, in part this is because the software in question is cumbersome to use and forces people to fit the way they work to the way the software requires them to work. There are significantly easier products to use. This, however, is not a function of proprietary versus open-source licensing. It's just a a matter of a poorly designed product.

No, the biggest problem, and yet also the easiest to solve, is the $3 million license fee. Any open-source solution would have allowed the PCAOB to spend nothing upfront and simply pay for support, thereby "amortizing" its risk. (Related to this, though not analogous, is the fact that smaller IT projects succeed much more often than large ones do, according to research from The Standish Group.)

Here are the PCAOB's two options:

1. Proprietary. Watch a few demos (hopefully representative of the product's actual capabilities (because you have no real way of knowing)), then fork out a huge pile of cash. Assume the cash has been incinerated the moment you give it away, because it won't buy you implementation or an actual solution to your problem. That costs extra, and it may never work.


2. Open source. Use the product for as long as you'd like. See if it works. If it does, and if you then want support (or, in some models, extra functionality), pay a fraction of what you'd pay to a proprietary software company. One hundred percent of that money is for support, such that the vendor has every incentive to ensure your continued happiness and to continue to invest in the innovation of its product and services. Pay extra for implementation, but often from a third party (and almost always much less) that doesn't have the same conflict of interest that in-house professional services might have. If the product doesn't work out, for whatever reason, you're out tens of thousands instead of millions.

Is there a real choice to be made here? Why would you ever take that proprietary software model over open source, assuming there is a roughly comparable open-source competitor to the proprietary product you'd normally choose?

Open source lowers IT risks in many ways, but in no way is this clearer than in how it dramatically lowers the cost of failure. It's common knowledge that many IT projects fail. As such, why not minimize the cost of failure as much as possible? It's much easier to try an open-source solution for $40K and discover that it's not a good fit for your enterprise than to spend 10 times that...only to discover the exact same thing.

This has nothing to do with the nature of the code itself. It has everything to do with the more customer-friendly business models that open-source companies use. Open source shifts the risk to the vendor, whereas proprietary software forces nearly all risk onto the shoulders of the IT buyer. (You don't get the product/code until you pay, and then you pay upfront.)

Open source is a better way of doing business. Maybe the PCAOB should try it.

NOTE: Someone has pointed out that the Public Company Accounting Oversight Board (PCAOB) is not a publicly funded organization. This is true. As the website says, "The PCAOB is a private-sector, nonprofit corporation, created by the Sarbanes-Oxley Act of 2002, to oversee the auditors of public companies in order to protect the interests of investors and further the public interest in the preparation of informative, fair, and independent audit reports." The central thesis of the entry holds, however: open source is a better way for IT buyers to do business. Whether it's someone's tax dollars or shareholder dollars, they shouldn't be wasted on proprietary licenses.

Disclosure: My employer, Alfresco competes with Documentum. But substitute Documentum with the proprietary company of your choice in the post above, and the story remains the same.

[ITsince67]

Standard operating procedure in government

This is a common story in state government.

I worked for the Commonwealth of Virginia's Dept of Information Technology (DIT) for over 10 years. State agencies could do projects with their own I.T. staffs, hire private companies, or..."give it to us". We often got projects after private companies turned down the project or ran into major problems.

Many projects are started without any real concept of what the requirements are. For some reason (they are never mentioned in public) someone in authority seems to pick a specific solution. Sometimes they work, sometimes they don't.

We (DIT) would be brought in and would be told by the politicans that we had to agree to the customer agency's estimate for time and budget they set. Many times, our staff was not trained in the software they had selected - and no training was provided.

At the Dept of Medical Assistance, one project escalated up to something like $83 M (back in 1990) dollars with no end in sight. Staff from the Auditor of Public Accounts came in and reviewed the project and said the project staff had no concept of where they were, where they were going, how to get there, or who was defining the requirements. Someone, not working on the project, was identified as the scrapegoat (would take the political fall for things not working). Instead, he jumped out a window on the 25th floor. DIT supplied info to the local paper that described him under a different name and job title. Even those of us who worked with him, could not identify him by the write-up in the paper. The Manager who selected him, was then promoted.

A Big 3 accounting firm once spent $1 M studying something, but said they could not define the requirements. I was given the same task. For $335,000 my team designed, built, and installed the application. What thanks did we get? I was told I was incompetence cause I came in at 10% over the originate project estimate. The real issue here, is "they" did not really want a working solution.

In some cases, the bureacrats do not really want an automated solution. To have one, would require they reduce their staff or show improvements in their operations. So,....in my opinion, they deliberately mis-defined the requirements so the developers cannot complete the project. If you cut the number of staff they supervise - shouldn't you cut their title and pay?

I left DIT and went to work for another agency. Found a report where each month, each site (142), reviewed file folders (up to 2,000/site) to submit a report to the central office. The information in the file folders was in the central computer system. It took a programmer less than 1 hour to write the report. Thanks we got? I got told I was stupid for producing the report - that now "new work" would have to be found for the staff in the field so they would have something to do for the last week of each month.

My response? I resigned.

[rock_va2000]

PCAOB does not run on tax $$

My friend PCAOB does not run on tax$$, its not even a Federal goverment body. Check the website, its a private entity.

[Matt Asay]

Very true - my mistake

I have corrected the original post. But it doesn't change the fundamental idea in the post: CIOs should not waste money on proprietary licenses. It significantly increases the cost of failure.