<- All posts

Two Upgrades, One Week: What the FAA Ground Stop Can Teach Us About Replacing Old Systems

·Cory Waddingham

On Friday afternoon, the FAA stopped every departing flight in the country. For a little over four hours, nothing took off. More than 5,000 flights were cancelled over the following days, and airlines are still repositioning planes and crews.

Let me say up front: nobody knows yet exactly what caused it. The investigation is ongoing, officials have given explanations that don't fully line up, and I'm not going to pretend to know more than they do. There are known unknowns here, and almost certainly some unknown unknowns too.

But even with the details still fuzzy, there's a lot we can learn from what we do know. Because what happened at the FAA is a very public version of something that happens in businesses every day: replacing old systems with new ones, sometimes with AI in the mix.

What we know so far

Two big changes happened at the FAA last week.

The first was a routine infrastructure upgrade. According to Transportation Secretary Sean Duffy, the FAA's longtime telecom contractor was switching lines from copper to fiber at a remote site when the outage began. Duffy said a router involved in that switchover began flooding other routers with connection attempts, and that older equipment, some of it around 25 years old and overdue for replacement since 2006, became overloaded. Communications at most of the FAA's 22 regional air traffic control centers were affected. Officials still don't know whether the router was defective or set up incorrectly.

The second was the nationwide rollout of SMART, a new AI tool that helps the FAA manage traffic flow. It pulls together around 200 streams of data, things like weather, flight paths, and staffing levels, and recommends reroutes to reduce delays. It started at three airports around Washington, D.C. in September, expanded to three more regional centers in October, and went live at all 22 centers on November 10. Three days before the ground stop.

Duffy says SMART played no role. The company that built it says the same. Homeland Security Secretary Markwayne Mullin has been more cautious, saying investigators "can't rule out anything" while they look into it.

Here's my take: I don't know which of these explanations will hold up, and for the purposes of this post, it doesn't really matter. The lesson is in the overlap.

When everything changes at once, nobody can tell what broke

Think about the position investigators are in right now. A major outage happens. Within the same week, a new AI system went nationwide and a contractor was rewiring part of the network. Now they have to work out which change caused the problem, or whether both played a part, or whether it was something else entirely.

That takes time. And while it's happening, every theory gets airtime, public trust takes a hit, and a promising new tool ends up under a cloud whether it deserves to or not.

This is one of the oldest lessons in managing technology: space out your changes. When you change one thing at a time and give it a chance to settle, problems are much easier to trace. When you stack changes on top of each other, even a small issue becomes a mystery.

For a business, this might look like not switching your accounting software the same month you roll out a new AI assistant to your finance team. Each might be a great idea on its own. Together, if something goes wrong, you won't know where to look.

Five habits that make upgrades safer

The FAA isn't the only organization juggling old systems and new technology. Most businesses I work with are doing the same thing on a smaller scale. Here are the habits that make the difference, each with a hint from last week's events.

1. Keep problems local. By Duffy's account, work at one remote site affected air traffic centers across the country. In any system, you want a failure in one place to stay in that place. Ask yourself: if this new tool or this upgrade misbehaves, how far can the damage spread? If the answer is "everywhere," it's worth adding some walls first.

2. Respect the limits of your old systems. Older systems often have quirks and limits that nobody remembers anymore. The person who set them up may be long gone. Before you connect something new, especially something that moves a lot of data, find out what the old system can actually handle. Equipment that's been quietly working for 25 years can surprise you the moment you ask it to do something different.

3. Roll out in stages, and give each stage time. This is where the FAA deserves real credit. SMART started small, in one region, and grew step by step. That's exactly the right pattern. The question worth asking is how long each step got to run before the next one. The biggest problems often show up after weeks of normal use, not days. A slower rollout can feel frustrating, but it's much cheaper than a fast one that goes wrong.

4. Involve the people who do the work. The union representing air traffic controllers said it wasn't involved in SMART's design, testing, or rollout. More than two dozen members of Congress also asked the FAA to pause the rollout over questions about safety testing and training. Whatever you think of those concerns, the people who use a system every day usually spot the edge cases first. Bring them in early, and the rollout gets better and they become champions for it. Leave them out, and you lose your best early warning system.

5. Know how to undo it before you need to. One thing that seems to have gone well: the FAA was able to switch off the problem function on the roughly 60 routers of that type across the country. That's a targeted fix rather than ripping everything out. For any change you make, know ahead of time how you'll turn it off or roll it back, and make sure someone can do it quickly.

What's different when AI is involved

Everything above applies to any upgrade. But AI adds a few things to think about.

First, the good news. SMART was designed so that a person reviews every recommendation before anything happens. Local facility leaders can accept or decline what it suggests. That's a smart design choice, and it's the right default for AI in any high-stakes setting. AI suggests, people decide. (We've written more about this in "How Do You Trust AI?".)

But AI tools bring their own challenges. They tend to connect to a lot of systems at once, since that's where their value comes from. SMART draws on about 200 data sources. Every one of those connections is a new dependency, and a new source of load on systems that may never have been built for it. AI tools can also behave less predictably than traditional software, which makes testing harder and staged rollouts even more important.

So treat adding AI like any other serious change to your operations: plan it, test it, roll it out gradually, and keep a hand on the off switch.

Upgrading is still the right call

It would be easy to walk away from Friday thinking new technology is risky and old systems are safe. I'd argue the opposite. Some of the equipment involved was supposed to be replaced 20 years ago. Duffy is now asking Congress for $50 billion to modernize the system. Every year an upgrade is put off, the risk quietly grows.

The goal is to keep changing things, carefully: one thing at a time, in stages, with the right people involved and a way back if something goes wrong.

We'll learn more about Friday as the investigation continues. In the meantime, it's a good moment to look at your own upgrade plans and ask whether you'd know what broke if something went wrong tomorrow.

If you're planning to bring AI into your business and want to do it without the drama, book a call with us.

Wondering what AI could do for your business?

Book a discovery call