NorthStar Digital · Free operating check

AI Incident Readiness Scorecard

Eight questions expose whether your team can identify, contain, communicate, and recover from a bad AI outcome without improvising under pressure. Choose the answer closest to your current operating reality.

No email requiredNo answers transmittedNo sensitive data requestedAbout three minutes

Contained synthetic storm above a black-glass incident strategy table

1. During an AI incident, who has authority to pause the workflow and coordinate decisions?A named primary and backup with a written escalation path.A named owner, but no tested backup or escalation path.Ownership depends on who notices first.Nobody is explicitly accountable.

2. Can an operator stop risky AI actions without taking the whole business offline?Yes; a tested kill switch or manual fallback is documented.A control exists but has not been rehearsed recently.Stopping it requires an engineer and improvised changes.There is no known containment control.

3. Can you identify the exact model, prompt, tools, data boundary, and configuration behind a disputed output?Yes; versions and actions are traceable for the agreed retention window.Most components are traceable, with known gaps.Logs exist, but reconstruction usually requires guesswork.No reliable reconstruction path exists.

4. Which AI actions require a human decision before they affect customers, money, access, or records?The boundary is explicit, enforced, and periodically reviewed.The boundary is documented but enforcement is partly manual.Teams apply judgment inconsistently.There is no defined approval boundary.

5. How would your team know the AI workflow is failing before a customer reports it?Named signals, thresholds, and an actionable alert path are tested.Monitoring exists but alerts are noisy or incomplete.Someone reviews logs or dashboards manually.Customer complaints are the primary detector.

6. When was the last realistic recovery or tabletop drill for this workflow?Within 90 days, with actions recorded and closed.Within the last year, or actions remain open.A document exists, but the team has never rehearsed it.No drill or recovery procedure exists.

7. What happens when a model provider, vector store, identity service, or other critical dependency fails?The workflow fails visibly into a tested safe state.Fallbacks exist, but behavior is not consistently tested.Failure behavior varies by component and operator.The workflow may continue incorrectly or fail silently.

8. Can the team produce one accurate incident timeline, decision log, and stakeholder update?Templates, evidence locations, and communication owners are known.A process exists but has not been tested end to end.Information is scattered across people and tools.Updates and evidence would be assembled ad hoc.

See the bounded next step Ask a scope question