Recovery, containment, authority
It's 4:15 pm on a Thursday. A product manager notices something wrong.
The AI system handling customer eligibility inquiries is returning answers that don't match policy. Not dramatically wrong—just wrong enough. A customer who should have qualified didn't. Then another.
She flags it to her manager. He's not sure this warrants shutting it down. He escalates to the platform team. They could disable it technically, but aren't sure they're authorized to. It's a revenue-impacting workflow. The product owner is on a flight.
Meanwhile, the system keeps running.
By the time someone with clear authority makes the call, 90 more minutes have passed. The system has processed another 400 interactions.
The question that surfaced wasn't technical. It was organizational:
Who can turn this off?
The Pattern
Most organizations can describe how to deploy AI. Fewer can describe how to stop it.
This isn't a gap anyone plans. It emerges from how AI systems get built—incrementally, across functions, without a single owner. By the time something is live, ownership is distributed. Distributed ownership works fine until something goes wrong and someone needs to act.
When leadership reviews what happened, the root cause is rarely "the model failed." Models fail. That's expected.
The governance failure is what happens next.
No shutdown owner. The system sits between IT, the business unit, and compliance. Each owns a piece. None owns the kill switch. When the question is "who can turn this off," the answer is a meeting.
Fear of overreaction. Operators hesitate. They're not sure the problem is severe enough. They worry about being blamed for disruption. So they escalate instead of act. Escalation takes time. Time is exposure.
No fallback exists. There's a deployment runbook. There's no rollback runbook. Turning it off means reverting to a manual process—but what process? Who staffs it? If the fallback requires improvisation, people hesitate to trigger it.
No threshold defined. What counts as "bad enough" to warrant shutdown? One wrong answer? Ten? A pattern? Without predefined criteria, every incident becomes a judgment call under pressure.
These aren't technology problems. They're governance problems: authority undefined, ownership diffused, recovery unplanned.
Why This Reaches Leadership
Most AI incidents don't start as board issues. They become board issues through a predictable sequence:
The system does something wrong. The organization takes too long to respond. Exposure accumulates. A complaint surfaces externally—regulator, press, lawsuit.
Then comes the accountability question: Who was in charge of stopping this?
If the answer is unclear, you no longer have a technical incident. You have a governance incident. Technical incidents are containable. Governance incidents raise questions about control, accountability, and culture.
That's the escalation pattern. And that's why shutdown authority matters as much as deployment authority.
What You'll Be Asked to Explain
When this reaches the board—and eventually, something will—they won't ask about the model. They'll ask about the organization.
Who made the call? Not "who could have." Who actually decided to shut it down? Was that person clearly authorized, or did they have to wait for permission?
How long did it take? From first detection to full containment. If the number is longer than it should be, why? Authority gap? Communication delay? Hesitation?
What happened in the meantime? How many customers affected? How many decisions made? What's the exposure—regulatory, legal, reputational?
These questions aren't adversarial. They're diagnostic. Leadership is trying to understand whether the organization has the reflexes to contain AI incidents—or whether governance is still catching up to deployment.
If you're not in the boardroom, you may still be the one asked to explain. This is the language they use. Knowing it protects you.
What You Need Ready
You don't want to answer these questions for the first time during an incident review.
A named shutdown owner. One person—and a backup—with explicit authority to disable the system without waiting for approval. Not the product owner. Not the tech lead. The person empowered to make the call under pressure.
A documented escalation path. Not "contact the on-call engineer." A specific sequence: who gets notified, in what order, with what decision rights. Include a time-to-containment target. If your escalation path doesn't have a clock, it's not a path—it's a suggestion.
A defined fallback. What happens when the system is off? How do customers get served? How do decisions get made? The fallback doesn't need to be elegant. It needs to exist.
Predefined shutdown thresholds. What conditions warrant disabling without further escalation? A single high-severity error? A pattern? A specific complaint type? Giving operators predefined criteria reduces hesitation.
The Real Test
Pick any AI system that touches customers or material decisions. Ask:
Who is authorized to disable it—by name?
What's the fallback while it's offline?
What conditions trigger shutdown without further approval?
What's the maximum acceptable time from detection to containment?
If you can answer all four, you have a kill switch.
If any answer requires a meeting, a judgment call, or "it depends"—you don't. You have an escalation process. Escalation processes fail under pressure.
The Bottom Line
Deployment authority is visible. Shutdown authority is often invisible—until it's needed.
Organizations invest heavily in getting AI systems live. They invest far less in the ability to stop them quickly.
That asymmetry is the gap. And when an incident exposes it, the question won't be "why did the model fail."
It will be "why couldn't you turn it off."
If no one is clearly empowered to stop a system, no one is truly accountable for what it does while it's running.
P.S. Pick one AI system in a customer-facing workflow this week. Ask the team to name—without looking anything up—who can shut it down, what the fallback is, and how long containment should take. If that conversation takes more than five minutes, you've found your gap. Fix it before someone else finds it for you.
Shameless plug:
Do you know someone who can benefit by learning the fundamentals of Artificial Intelligence (AI) and Machine Learning (ML)? You are in luck!
I have created a fundamental course on AI/ML where I explain this complex topic is the most simply way - some of my students calls it “oversimplifying”!
Click on this link and gift them the course - and yes, they do not need technical background. I mean it - else they get their money back! Guarantee!
{{ address }}
Unsubscribe · Preferences
