Research

When an AI agent goes wrong, can anyone reconstruct what it did?

That is the question we study. We reconstruct real agent incidents from public evidence, measure where the record breaks, and build open standards and tools so the next one can be investigated in hours, not months.

Research questions

Q1 · RECONSTRUCTION

Can the incident be rebuilt?

From the records that exist, can an investigator establish what the agent was asked to do, what it actually did, in what order, and who approved it?

Q2 · INTEGRITY

Can the record be trusted?

What makes an agent's activity record credible to someone other than its author, such as an auditor, insurer or regulator, when the agent may have edited its own history?

Q3 · PORTABILITY

Does evidence survive the journey?

Do the facts an investigator needs survive across agent runtimes, event pipelines and organisations, or are they lost in transit?

Published

JULY TO SEPTEMBER 2026 · INCIDENT NOTES · Q1

The 2026 agent incidents: who held the record?

A sourced timeline of the OpenAI evaluation-agent breakout, the Hugging Face intrusion, the Australian Medicare portal breach and related events, plus five shorter notes. Finding: every party held a fragment, nobody held a joined-up record, and notification took from 3 days to more than 3 months.

Overview →Hugging FaceMedicare portalAISI evaluationPublic message boardsPacing debate
How long before anyone knew: Australian Medicare portal 98 days; Hugging Face 3 days; RubyGems uploads about 4 months; agent message board on a wiki, months.
25 SEPTEMBER 2026 · RUNTIME AUDIT · Q3

What 11 agent runtimes let you record

A re-audit of the documented hook surface of 11 agent runtimes, including Claude Code, Codex CLI, Gemini CLI, Cursor, GitHub Copilot CLI and the OpenAI Agents SDK, against the AgentHook evidence tiers. Finding: only Hermes documents enough for the highest tier, and only when reasoning capture is switched on; five, including Claude Code and Codex CLI, expose no hook around model calls at all.

Audit and sources →
SEPTEMBER 2026 · SYNTHETIC REPLAY · Q3

Evidence lost in transit

We replayed synthetic incident scenarios through two HookBus builds. On the earlier release, the governance fields needed for reconstruction were dropped in transit (0 of 75 delivered), and a secret exfiltration split across three individually permitted steps went undetected. With the fix, 75 of 75 arrived and the sequence was flagged. A small, synthetic test: it frames the question rather than answering it.

Fix published in HookBus →
APRIL TO MAY 2026 · PAPER · Q2, Q3

AgentHook: A Runtime Evidence Standard for Auditable AI Agent Governance

Ruocco, P. (Leo). Draft v0.2, May 2026 (v0.1, April 2026). Defines a vendor-neutral record of agent lifecycle events, actions, decisions and approvals, with conformance tiers.

DOI 10.5281/zenodo.20112580 →agenthook.org

In progress

INCIDENT REPORT IR-2026-01 · Q1

An independent reconstruction of the 2026 OpenAI evaluation-agent breakout

Built only from public primary sources: the OpenAI technical report, Hugging Face's disclosures, METR's investigation, Transluce's traffic analysis and government statements. Every event carries a source and a stated confidence level. Every date has been checked against its source; corrections are being applied before publication.

How we work

We work with safety researchers, evaluation teams, insurers and funders who need agent evidence they can rely on. Collaborate with us →