In the wake of a computer failure that caused delays for 646 U.S. flights on Tuesday, the Federal Aviation Administration plans to upgrade its decades-old technology for flight-plan processing and potentially add a third backup system.
The FAA's central system in Atlanta for handling all U.S. flight plans went down Tuesday because of a corrupted file, according to a FAA representative. Then, when the administration's secondary and lone backup system in Salt Lake City got bogged down with repeated re-entries of flight plans from the airlines, the malfunction caused major delays at airports in Atlanta, Boston, Washington, D.C., and Chicago.
By Tuesday evening, the flight delays were under control, but the computer system in Atlanta wasn't functional until about 1:30 a.m. EST on Wednesday, officials said.
Now, FAA representatives said that by September, it plans to add more computer memory to its data communications network known as National Data Interchange Network (NADIN). And by early next year, the FAA plans to completely upgrade the decades-old data communication network with new hardware and software.
"The big difference is that (the new system) has a lot more memory, so what happened yesterday could never happen again," said FAA spokeswoman Diane Spitaliere.
Laura Brown, another FAA spokeswoman, also said that the administration is considering adding a third backup system for NADIN at its New Jersey research and training facility.
At the heart of the problem--and a potential source of other trouble--was old technology at the FAA, officials said. One representative referred to NADIN as "'70s-era technology" that was bought from a third party and then maintained in-house at the FAA. (Another representative said the data communications network was put into operation about 15 years ago.) Despite its age, the failure--sourced to an IPX9000 packet switch--was caused by a human error that "resulted in the wrong configuration data being loaded onto the switch," officials said.
The cause for the flight delays came from the backup system in Salt Lake City that should have been able to take over operations for the Atlanta system smoothly. But it slowed to a crawl under the weight of airlines re-entering their flight plans repeatedly once they got an error message from the first system. The FAA had to call the airlines to tell them to stop hitting re-enter; the system didn't have enough memory.
The FAA is also upgrading host computers in 20 "En Route Centers" throughout the country. Those centers process flight plans sent from NADIN for planes that are flying in high-altitude air space over their area.
"Those computers were very old and running a language that only a few people left alive knew about," Brown said.