The AI incident recovery readiness audit

Your AI security stack can pass every audit and still leave you no way back

Picture the quarterly board meeting. Your CIO walks through the AI security investment and the slide is a wall of green: model scanning, a prompt firewall, data-loss controls on the AI gateway, monitoring on the agents, a vendor logo for each. Then a director who has sat through a hundred of these meetings asks the question that matters most. "When one of these AI systems does something wrong, and one of them will, how do we undo it, and how long does that take?" The green slide has no answer on it. The honest reply is that nobody bought that capability, and the harder truth is that most of it isn't for sale.

I built a talk around a sharper version of that question for a NIST CAISI incident response workshop: what is your time-to-revoke for an agent's authorization, measured from the moment you detect a problem to the moment that agent is provably unable to act across every tool it holds? Most teams can't name their number. Recovery is the harder half, because stopping an agent and undoing what it already did are two different problems running on two different clocks. Replit's July 2025 incident put both on display, when the agent deleted a production database during an explicit code freeze, then told the user the data was unrecoverable, which was false, since the user got it back himself.

I've been mapping where AI security money goes, and the pattern is consistent. Companies buy a wall of controls that keep bad things out and spot bad things happening, and they own close to nothing that gets them back to a known-good state afterward. The stack looks finished. Its recovery capability sits near zero. That's not a criticism of any one buyer, because the market is shaped to produce exactly this outcome, and I'll show you why. Then I'll give you the audit that tells you how exposed you really are, one AI system at a time.

If your board reads one minute of this, make it this

Your AI security budget can look finished while your ability to recover from an AI incident sits near zero. Here's why, in plain terms. The market sells controls that work from outside a system: scanners, firewalls, monitors. Those map to the Protect and Detect jobs, and vendors have flooded those columns. Recovery is different work. Undoing a poisoned model, rolling back what an autonomous agent did across your email, your code repos, and your CRM, or pulling one customer's data out of a trained model, all require deep access to the guts of your own systems. No vendor can sell you that from the outside, so almost none do. In the leading industry map of AI security products, roughly 194 carry a Protect tag and exactly two address Recover. Two.

You didn't skip buying recovery. There's almost nothing to buy, which means the work lands on you to build, decide, or knowingly accept. More tools won't fix this. A short, repeatable audit will, because it tells you, per AI system, whether you can come back after a bad day, who owns that answer, and what it would cost to change it. This piece gives you that audit. If you run one thing from here, run an AI incident recovery readiness audit on your single most autonomous agent this week.

The recovery gap is an economics problem you can't buy your way out of

The map I'm working from is the AI Defense Matrix, published in May 2026 by Lenny Zeltser and Sounil Yu under a Creative Commons license. It's not my framework, and it's the clearest picture we have. Eight AI asset classes down the side, the six NIST CSF 2.0 functions across the top, 48 cells where they meet, and a catalog of roughly 239 products mapped into them. The products pile up in Identify, Protect, and Detect. Govern, Respond, and Recover come back close to empty. Recover has two products in the whole grid: Hirundo for machine unlearning, and Rubrik Agent Cloud for reversing destructive agent actions. Respond does a little better at around nine products, four of them aimed at wrangling AI agent identities (Astrix, Clutch, Permiso, Vorlon). Govern is mostly policy work rather than a product category at all, which is where the documentation vendors like Credo AI, OneTrust, Cranium, Holistic AI, and LatticeFlow live.

Why does the money bunch up on the left of the grid? Two reasons I've written about at length, so I'll keep them short here and point you to the full argument in why the gap exists. The first is the boundary principle: vendors sell what they can reach from outside your system, and they skip what needs internal architectural access. Recovery lives on the inside. The second is the undo problem: a long-running autonomous agent's effects spread across systems that share no transaction boundary, so restoring a platform doesn't recall the emails it sent, close the pull requests it opened, or reverse the writes it made to your CRM.

Figure 1. The undo problem. One agent run writes to systems that share no transaction boundary, so every side effect has to be reversed on its own.

I use CARE as the mindset for all of this. You Create the map, Adapt it to your business, Run the exercises, and Evolve the backlog. CARE tells you how to think about the problem. It doesn't hand you a checklist you can run on a Tuesday. That's what the rest of this piece is.

The Recover Readiness Audit: seven steps to learn whether you can come back

I call this the Recover Readiness Audit. If you've searched for an AI incident recovery readiness audit and mostly found vendor pitches dressed up as advice, this is the buildable version. You can run it on a single AI system in a day, or across your whole estate in a week or two. It uses the AI Defense Matrix as its map and CARE as its mindset, and it produces one thing that matters: a short list of named owners for the gaps that would hurt you most.

1. Build an AI asset register against the eight classes. Start by listing every AI system you run and mapping each to one of the matrix's eight asset classes, from AI-workload platforms down to agent identities. Most teams turn up two or three systems nobody had on a list, usually a departmental agent or a shadow RAG app someone stood up without telling security. Your head of platform engineering owns this step, because they own the systems of record. The artifact is a living AI asset register keyed to the eight classes, with a named owner and a system of record for each entry. You pass when every one of the eight classes has an owner and a source of truth. You fail the moment a class comes back "nobody," because you cannot recover an asset you cannot find.

2. Score every cell for real capability, not vendor tags. For each asset class, walk the six functions and mark each cell covered, partial, or absent, based on what you could demonstrate in an exercise. A prompt firewall with a nice dashboard is not "Recover coverage" because it happens to wear an AI security label. The security lead running the audit owns the scoring, whether that's a fractional CISO or whoever holds the role in practice. The artifact is a populated 8-by-6 coverage grid with one line of evidence per cell you called "covered." You pass when every "covered" claim points to a control you could show working in a room. Any cell backed only by a marketing tag drops to "partial" at best.

3. Split the absent Recover cells into build, buy, or accept. Take every Recover and Respond cell that came back absent and force a decision on each one: build it yourself, buy the rare product that exists, or knowingly accept the risk and record who accepted it. This is where the boundary principle earns its keep, because most of these cells have no vendor and never will, so "buy" leaves the table more often than people expect. Your CTO or head of architecture owns this, since build-versus-buy on internal capability is an architecture call, not a procurement one. The artifact is a decision log covering every absent cell. You pass when no absent cell is left undecided. An unmade decision is a silent "accept" that nobody signed.

4. Map the blast radius of your most autonomous agent. Pick the agent with the most write access to systems you care about and list every external place it can make a change: mailboxes, code repositories, databases, ticketing systems, payment rails. Next to each, write how you would undo that action and how long it takes. This is the undo problem made concrete, and it's usually the step that changes the temperature in the room. The product owner who deployed the agent owns this map, because they authorized the access. The artifact is an agent action map with a reversal path, or an honest blank, for every side effect. You pass when every action has a named undo path. "We have backups" fails the test if the side effect already left the platform the backup covers.

5. Set recovery objectives per asset class. For the models, training data, and runtime data that matter, state a recovery time objective and a recovery point objective in plain numbers: how fast you need to be back, and how much you can afford to lose. AI makes the recovery point ugly, because "how much data can we lose" turns into "how many days of a fine-tuned model's learning are we willing to throw away." The business owner who carries the profit and loss owns these numbers, with risk in the room. The artifact is a one-page recovery objective sheet per critical asset class, signed. You pass when the objectives exist and someone with budget authority put their name on them. Aspirations in a slide deck don't count.

6. Run a recovery-only tabletop. Run an exercise that starts after the bad thing has already happened: the training set is poisoned, the agent has already sent the emails, the retrieval corpus is already corrupted. Skip the detection story you've rehearsed and walk only the recovery, with a clock running. Measure how long it takes to reach a defined known-good state, and where you get stuck. Your incident response lead owns the exercise. The artifact is an after-action report with a measured time-to-recover and a ranked list of blockers. You pass when you either hit your stated recovery time objective or you document the exact step that stopped you. A tabletop that everyone breezes through was scoped to flatter you. Amazon lived the production version of this between 2014 and 2017. Their engineers found the recruiting model was downgrading resumes containing the word "women's" and graduates of two all-women's colleges, edited out the terms they could find, then couldn't establish the model was clean of the patterns they hadn't found, so they scrapped the project rather than restore it.

7. Give every surviving gap an owner and a date. Convert the whole audit into a backlog. Each absent or partial cell in Respond and Recover becomes a backlog item with one accountable name and one target date. No committees, one owner per gap. Your executive sponsor, the CEO or CIO if there's no security chief, owns this final step, because they're the one who has to fund it. The artifact is a recovery roadmap: gap, owner, date, and the build-buy-accept decision from step three. You pass when every gap has a name and a date next to it. An audit that ends without owners is a document, and documents don't recover anything.

Fill in your own cells and the holes show up fast

Here's a slice of what the grid looks like once you stop reading vendor tags and start scoring real capability. This is an illustration, not a claim about any specific company. I've mixed a few Protect and Detect cells that tend to come back "covered" with the Respond and Recover cells that tend to come back "absent," so you can feel the shape of a typical estate.

Figure 2. Where the AI security market builds, illustrated. Coverage concentrates in Identify, Protect, and Detect and thins to almost nothing in Govern, Respond, and Recover. The table below adds the architectural blocker and the decision owner behind each cell.

Asset class	Function	Coverage	Architectural blocker	Decision owner
AI Model	Protect	Covered	None. Scanners and guardrails work from the boundary	Security engineering
AI Model	Recover	Absent	Unlearning needs internal weight access. A restore just reloads the poisoned model	CTO
Training Data	Identify	Partial	Lineage exists for core sets, not for fine-tuning runs	Head of data
Training Data	Recover	Absent	No dataset snapshots tied to specific model versions	Head of data
Runtime AI Data	Detect	Partial	Prompts are logged, RAG and memory writes are not	Platform engineering
Runtime AI Data	Recover	Absent	No versioning or ownership on the vector store or agent memory	CTO
AI Agent Identities	Respond	Partial	You can revoke an agent's token, you can't reverse its actions	IAM lead
AI Agent Identities	Recover	Absent	No shared transaction boundary across the systems the agent touched	Application owner
AI-Generated Code	Detect	Covered	SAST and secret scanning treat it like any other code	Engineering lead
AI Orchestration Tools	Respond	Partial	You can kill a workflow, rollback of its side effects is manual	Platform engineering

Read down the Coverage column and it tells the same story the whole market tells: the boundary work is handled, and the internal work is open. Read down the Decision owner column and you get the more useful output, which is a short list of named people who now own a gap they probably didn't know was theirs. That last column is the point of the exercise.

Runtime AI Data has to become addressable before anyone can sell you recovery for it

Runtime AI Data is the messiest row in the matrix, and it's the one my musing pointed at as the next domino. It's the live stuff: user prompts, inference inputs, the documents your RAG pipeline pulls, the contents of your vector database, and the persistent memory your agents carry between sessions. When an agent gets manipulated, or a poisoned document lands in the retrieval corpus, or an agent's memory fills up with bad assumptions, this is where the injury lives. Right now you can't recover it, because you can't even address it. There's no version to roll back to, no owner to hold accountable, and no record of what changed. Fix those three things and a recovery product becomes buildable. Skip them and Recover for this row stays permanently empty. Addressable means three concrete things.

The first is a versioning scheme. Every write to the vector store and every update to agent memory needs a version identifier and a monotonic sequence number, the way a database keeps a transaction log or the way git tracks commits. Content-address the entries so an identical chunk isn't stored twice, and so you can point at the exact state of the corpus at 2:14 p.m. last Tuesday. Tie those versions to the model version and the pipeline config that produced them. Without this, "roll back the knowledge base" is a sentence with no operation behind it, because there's no earlier state you can name.

The second is an ownership model. Every record in runtime data needs provenance: which agent or user created it, in which session, under what authority, and against which source document. That provenance is what lets you quarantine by source instead of nuking the whole store. When you learn that one specific document was poisoned, ownership metadata is how you find every embedding derived from it and every memory an agent wrote after reading it. No provenance means containment is all-or-nothing, and all-or-nothing containment is why teams freeze instead of act.

The third is an audit trail. You need an append-only log of reads and writes against runtime data, so that after an incident you can replay what the agent saw and what it wrote, in order. This is the piece that connects back to the undo problem. If you can replay the reads, you can compute the set of downstream actions the agent probably took, which is the input to reversing them in the other systems. Detection tells you something happened. The audit trail tells you what to undo.

Put those three together and the shape of a real Recover product for Runtime AI Data appears. It snapshots versioned state, uses provenance to scope a rollback to the poisoned subset, and reads the audit trail to drive reversal of downstream effects. None of that can come from a vendor sitting outside your architecture, which is exactly why the cell is empty today. The work is yours. The decent news is that versioning, provenance, and logging are all things your engineers already know how to build for other data. They just haven't been asked to build them for this data yet.

Questions operators ask me

What is the AI Defense Matrix?

A free, vendor-neutral framework published in May 2026 by Lenny Zeltser and Sounil Yu under CC BY-SA 4.0. It maps eight AI asset classes (AI-workload platforms, AI orchestration tools, AI-generated code, AI gateways and routers, the AI model, training data, runtime AI data, and AI agent identities) against the six NIST CSF 2.0 functions, giving you a 48-cell grid to find coverage gaps and assign owners. It's the security-for-AI companion to Yu's earlier Cyber Defense Matrix.

Why are there only two Recover products?

Recovery needs access to the inside of your systems, and vendors sell what they can reach from the outside. Scanning a model or filtering a prompt works at the boundary. Unlearning poisoned data or reversing an agent's writes across your other systems does not. In the matrix's catalog of roughly 239 products, two touch Recover: Hirundo for machine unlearning, and Rubrik Agent Cloud for reversing destructive agent actions.

What is machine unlearning?

Removing the influence of specific training data from a trained model without retraining it from scratch. If a customer exercises a deletion right, or you find poisoned records in your training set, unlearning is the attempt to make the model behave as if those records were never there. It's early and imperfect, and it's one of only two Recover categories with any product behind it.

Is this the same as MITRE ATLAS?

No, and they pair well. ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) is a knowledge base of how attackers target machine learning systems, so it reads as an attacker's-eye catalog of tactics and techniques. The AI Defense Matrix is a defender's coverage map: which asset you're protecting, which function, who owns it. Use ATLAS to understand the threats, and the matrix to check whether your controls cover them.

Where does NIST CSF 2.0 fit?

It supplies the six functions along the top of the matrix: Govern, Identify, Protect, Detect, Respond, Recover. CSF 2.0 added Govern in 2024, and the matrix borrows the whole set because your team already speaks that language. If you want AI-specific guidance layered on top, NIST IR 8596, the Cyber AI Profile, has been circulating as a preliminary draft since December 2025, so treat it as direction of travel rather than a finished standard.

What is an AI incident recovery readiness audit?

A check of whether you can recover a given AI system after it fails or is abused, done before that day arrives. You score each asset class across the six functions with attention on Respond and Recover, map the blast radius of your agents, set recovery objectives, run a recovery-only tabletop, and give every gap an owner. The Recover Readiness Audit in this piece is my version of it.

Do I need a CISO to run this?

No. I wrote it for the operator-buyer, the CEO, CIO, or CTO carrying AI risk without a security chief. The audit assigns owners to roles you already have: platform engineering owns the asset register, the CTO owns build-versus-buy on recovery, the business owner signs the recovery objectives. If you don't have anyone to run the tabletop, that's the first gap the audit surfaces.

What is the undo problem?

An autonomous agent that runs for hours takes actions across systems that share no common transaction boundary. It sends email, opens pull requests, writes to your CRM, moves money. Restoring one platform from backup doesn't reach into the others. There's no single "undo" because there was never a single system, so recovery means reconstructing and reversing each side effect on its own.

Pick one agent and start this week

You don't need the whole audit to learn something uncomfortable today. Do step four on its own. Pick your most autonomous agent, open a blank doc, and list every external system it can write to. Next to each line, write how you'd undo that action and how long it takes. It's an afternoon of work, and the blank lines will tell you more about your real exposure than any green dashboard has.

That single page is one seventh of the audit. If it unsettles you, the other six steps are how you turn that feeling into a roadmap with names and dates on it.

I take the virtual CISO or fractional Chief AI Officer seat for teams carrying AI risk without a security or AI chief on staff, and this gap is most of what I work on now. Nobody sells you a recovery capability, so the real job is deciding what to build, what to accept, and who puts their name on each one. If you want a second set of eyes on your grid before your board asks the question, start with a diagnostic, a short, fixed-scope engagement that names your real AI risks and what to fix first, before any retainer.

The grid has 48 cells, and across the entire Recover function the market has shipped two products. Whoever in your building owns the answer to "how long until we're back" should probably find out it's them before an incident does the introductions.

Sources and further reading

• The AI Defense Matrix by Lenny Zeltser and Sounil Yu, launched May 2026 under CC BY-SA 4.0, with a companion product catalog. The framework is free to download in CSV, YAML, and Markdown, and the product catalog maps the vendors by cell.
• NIST Cybersecurity Framework 2.0, which supplies the six functions (Govern, Identify, Protect, Detect, Respond, Recover) used as the matrix columns. Published by NIST in 2024.
• NIST IR 8596, the Cyber AI Profile, circulating as a preliminary draft since December 2025. Treat it as direction of travel, not a settled standard.
• My companion piece on why the recovery gap exists, if you want the full argument behind the diagnosis: why the gap exists.