The Executive Guide to AI Incident Response and Recovery

Treat AI incidents as operational failures, not model curiosities

An AI incident is any event in which an AI-enabled system causes, enables or materially increases harm. That includes discriminatory recommendations, fabricated legal or medical claims, disclosure of confidential data, unauthorised transactions, prompt-injection attacks, runaway agent actions and outages at a model or tooling provider. The threshold should be impact, not novelty. A customer-facing chatbot inventing a refund policy is an incident even if the underlying model is behaving exactly as designed; a benchmark regression with no production effect usually is not.

Executives should establish one incident process that connects AI specialists with security, privacy, legal, operations, communications and the business owner. Creating a separate, research-led track for AI failures invites delay because the team first debates whether an event is a hallucination, a jailbreak or a data-quality problem. Customers and regulators care less about taxonomy than consequences. Use the organisation’s existing command structure, evidence standards and escalation channels, then add AI-specific expertise and controls.

Preparation begins with an inventory of deployed models, agents, prompts, retrieval sources, tools, data flows and vendors. Each system needs a named business owner, technical owner, risk tier, permitted actions and emergency shutdown method. Record which applications can send emails, query personnel files, issue credits or write to production databases. If leaders cannot identify those capabilities within minutes, they cannot credibly contain them during an incident.

Define severity before pressure distorts judgement

A four-level severity scale gives responders a common language. Severity 1 should cover catastrophic or potentially catastrophic events: large-scale disclosure of regulated data, credible threats to safety, autonomous actions causing major financial loss, or systemic harmful decisions affecting thousands of people. Severity 2 can cover serious but bounded harm, such as exposure of dozens of customer records, a discriminatory screening pattern in one region, or an agent sending unauthorised messages. Severity 3 covers limited, reversible impact; Severity 4 covers near misses and policy breaches without confirmed external harm.

Set measurable triggers wherever possible. Examples include more than 500 affected users, losses above £100,000, exposure of special-category personal data, compromise of model credentials, actions across multiple production systems, or an outage lasting more than two hours in a critical service. Numbers prevent a product team from downgrading an event to protect a launch, but they should not replace judgement. A single incorrect medical recommendation may warrant Severity 1 treatment even when no financial threshold is crossed.

Severity must determine response obligations. A Severity 1 event might require an incident commander within 15 minutes, executive and legal notification within 30 minutes, continuous staffing, hourly situation reports and board notification. Severity 2 may require mobilisation within one hour and updates every four hours. Assign authority in advance: the incident commander controls operations, the business owner assesses customer impact, security preserves evidence, legal interprets notification duties, and communications manages external statements.

Contain harmful outputs without destroying the evidence

Containment should reduce exposure faster than the team seeks certainty. For a harmful-output incident, disable the affected feature, route traffic to a safer model, narrow permitted topics or require human approval before publication. For data leakage, revoke credentials, block suspect prompts and connectors, isolate retrieval indexes, suspend logging pipelines that capture sensitive content and rotate exposed secrets. For a runaway agent, remove tool permissions, halt queued jobs and impose transaction, rate and spending limits at the execution layer rather than relying on a revised system prompt.

The trade-off is availability. Shutting down an entire customer service platform may prevent further disclosure but also leave 200,000 customers without support. A graduated response is often better: switch from generative answers to approved templates, disable file uploads, remove access to account data, or limit the service to authenticated users. Design these degraded modes before an incident. A kill switch that has never been tested is merely a hopeful interface.

Preserve evidence while containing the system. Capture prompts, outputs, model and prompt versions, retrieval results, tool calls, user identifiers, policy decisions, timestamps and configuration changes. Snapshot relevant indexes and logs using access-controlled storage, and document who collected each item. Do not allow teams to ‘fix forward’ by editing prompts or deleting conversations before evidence is secured. Reproduction may be difficult because model providers change weights, safety layers and routing without notice.

Respond differently to the four dominant failure modes

Harmful outputs require rapid scope analysis: which prompts triggered them, whether outputs reached users, whether affected groups experienced disparate treatment, and whether the problem persists across languages or channels. Data exposure demands a privacy and security investigation covering what information left the boundary, who received it, whether the provider retained it and whether contractual or statutory notification clocks have started. In the UK, organisations may need to notify the Information Commissioner’s Office within 72 hours of becoming aware of a qualifying personal data breach.

Runaway actions demand transaction-level containment. An agent that mistakenly orders stock, changes prices or emails suppliers can multiply damage in seconds. Revoke its delegated authority, freeze downstream workflows, identify completed and pending actions, and establish which actions are reversible. Recovery may involve cancelling orders, restoring records from snapshots and contacting counterparties. The central lesson is architectural: high-impact tools need allowlists, dual approval, idempotency controls, hard monetary caps and an immutable activity ledger.

Vendor outages and degradations require a different posture. The supplier may return successful API responses while quality, latency or safety deteriorates. Monitor semantic performance as well as uptime: refusal rates, citation accuracy, tool-call errors, toxicity indicators and cost per completed task. Maintain a tested fallback, such as a secondary provider, a smaller internal model or a non-AI workflow. Multi-vendor resilience costs more and can introduce inconsistent behaviour, so reserve it for services whose downtime or degradation exceeds the cost of duplication.

Communicate facts, decisions and uncertainty

Incident communications should answer five questions: what happened, what is affected, what has been stopped, what users should do and when the next update will arrive. Separate confirmed facts from working hypotheses. ‘We are investigating reports that the assistant displayed information from other customer accounts’ is more credible than claiming no breach occurred before logs are reviewed. Avoid blaming a model provider unless evidence supports the claim; the organisation operating the product remains accountable to its customers.

Internal updates should follow a fixed format: severity, incident owner, business impact, timeline, current containment, unresolved risks, decisions required and next update time. A concise hourly report is more useful to executives than a stream of technical chat. Keep a decision log recording who authorised shutdowns, customer credits, vendor escalation and regulatory notifications. This protects accountability and helps later reviewers understand choices made with incomplete information.

Prepare message templates for customers, employees, regulators, partners and the press. Templates should not predetermine conclusions, but they can establish approval routes and required facts. Legal review is essential, yet excessive caution can produce evasive statements that deepen reputational damage. If users need to reset credentials, disregard an AI-generated instruction or verify a transaction, say so plainly. Speed matters most when communication can prevent further harm.

Recover through controlled validation, not a hopeful restart

Recovery is the restoration of a safe business service, not merely the return of an API. Define exit criteria during response: the exploit path is blocked, exposed credentials are rotated, unauthorised actions are reconciled, affected data is corrected, monitoring is active and accountable owners accept residual risk. For a harmful-output incident, test representative prompts, adversarial variants and affected demographic or language groups. A patch that succeeds on ten hand-picked examples is not sufficient evidence.

Use staged restoration. Begin with internal users, then perhaps 1 per cent of production traffic, followed by 10, 50 and 100 per cent as metrics remain within thresholds. Keep human review and stricter rate limits during the early stages. Compare quality, safety, latency and cost against the pre-incident baseline. Establish automatic rollback conditions, such as more than one confirmed cross-account disclosure, a 2 per cent tool-execution error rate or a threefold increase in unsafe-output alerts.

Recovery also includes people and records. Correct customer accounts, reverse transactions, provide compensation where appropriate and ensure support staff have accurate scripts. Update the asset inventory, data-protection assessment, model card, operating procedures and vendor risk record. If the business accepts a temporary limitation, such as disabling document uploads for 30 days, name the executive owner and expiry date so an emergency compromise does not become an unmanaged permanent state.

Turn postmortems into funded controls

Hold the postmortem once immediate risk has passed, usually within five to ten working days. Make it blameless but not consequence-free: focus on conditions and decisions while assigning owners to corrective work. Reconstruct detection, escalation, containment and recovery. Quantify customer impact, financial loss, staff hours, service interruption and regulatory exposure. Ask why controls failed at each layer, including product design, permissions, monitoring, supplier management and executive oversight.

Distinguish root causes from triggers. A prompt injection may trigger an agent to disclose data, but the root causes could include excessive connector permissions, untrusted content entering the context window, missing output filtering and absent egress controls. Corrective actions should therefore span prevention, detection and damage limitation. ‘Improve the prompt’ is rarely an adequate response to an incident involving privileged tools.

Track actions like audit findings, with an owner, budget, due date, verification method and risk-acceptance route. Report recurring patterns and overdue items to the executive risk committee. Useful programme metrics include mean time to detect, time to containment, percentage of critical systems with tested kill switches, percentage with vendor fallbacks, and recurrence within 90 days. The board’s role is not to review every prompt; it is to ensure that high-impact AI systems have bounded authority, observable behaviour and a rehearsed path to safe recovery.