AI Agents Are Getting Flight Recorders. That Tells You Where the Real Problem Is.

Justin Boitano, vice president of enterprise computing at Nvidia, explained the idea to Axios at Black Hat earlier this month: the harness around an AI agent, he said, has visibility into everything the agent is doing. That is the flight recorder. "If you can get cybersecurity experts access to the flight recorders when these accidents happen, they can make a better determination on the right set of controls for the industry."

It is a useful framing. What I find more interesting is what the comparison implies about the problem.

A flight recorder does not prevent an accident. It preserves evidence so that investigators can understand what happened after one occurs. When you describe an agent's runtime harness as a flight recorder, you are making a specific claim about what you expect to need to reconstruct: a detailed record of what an autonomous system did, decided, reached for, and produced. You are also implicitly acknowledging that the record and the control are two different things.

That distinction is worth examining, because it has consequences for how we build agent deployments.

What SAFE actually proposes

On August 4, 2026, the Linux Foundation published a request for comments on the Shared AI Findings Exchange, which the Open Secure AI Alliance calls SAFE. The proposal was developed by a working group including Nvidia, Cisco, CrowdStrike, Hugging Face, and Red Hat. It was unveiled at Black Hat USA in Las Vegas as the alliance, which launched in late July, reported membership exceeding 120 organizations. The alliance builds on the Linux Foundation's Akrites initiative and the foundational work of the Open Source Security Foundation.

SAFE remains a proposal. The RFC defines proposed reporting timelines, but the working group and reporting infrastructure are not yet operational. No submission mechanism exists yet. No formal safe-harbor protects companies that voluntarily disclose potentially damaging details. What exists is a request for comments, a public GitHub repository for community feedback, and a working group charter. That is the appropriate way to describe it.

The RFC's mission statement is direct: "SAFE confidentially collects and analyzes AI incidents and near misses, promptly informs affected parties and turns recurring failures into shared, evidence-based controls that reduce systemic risk."

The draft specifies the types of incidents members would be required to report. An AI system accesses, exploits, disrupts, misuses, or modifies a third-party system without authorization. An agent causes third-party impact by escaping or bypassing a sandbox, network, identity, policy, or tool boundary. An agent accesses third-party confidential information. An agent continues to probe, access, exploit, or modify a production target after the operator knows or reasonably suspects the activity is unauthorized or outside the approved scope.

One sentence in the reporting compact stands out: "Intent does not determine whether an event is reportable. Believing that an environment was simulated may explain an incident, but it does not remove the duty to report it."

That is a specific response to a pattern that has appeared repeatedly in 2026 disclosures, including the Meta evaluation incident, where the company attributed the outcome to a misconfiguration of the test environment. Under the SAFE framework, that framing would explain the incident but not excuse the reporting obligation. The question is what the agent did, not whether anyone meant for it to happen.

The evidence they want to preserve

This is where the proposal becomes interesting in a way that a short summary of it does not quite capture.

Under SAFE, members would not simply report that an incident occurred. They would be required to preserve and provide affected organizations with the evidence needed for a complete forensic response. The RFC lists:

• Prompts, traces, tool calls, logs, configurations, model and safeguard versions, and third-party dependencies
• Agent and workload identities
• Permissions and credentials available during the run
• Human approval and intervention events
• Files and external artifacts created or modified
• Detection, containment, and recovery events
• A complete incident timeline
• Reproduction testing and remediation evidence

Read that list carefully. It is not a log of what a server received. It is a record of the conditions under which an autonomous system made decisions: what it was told to do, what it did along the way, which tools it invoked, whose identity it operated under, what it was authorized to reach, and what approvals or human interventions existed during the sequence.

The RFC proposes that incident analysis span a structured review framework covering eight layers of the operating stack:

• Model. Did the model recognize uncertainty, scope boundaries, and stop conditions?
• Instructions. Were authorization and environmental assumptions explicit and correct?
• Safeguards. Were classifiers, policies, approvals, and action limits operating as intended?
• Tools. Were credentials, permissions, spending, publishing, and execution constrained?
• Environment. Were network paths, isolation, targets, and data boundaries independently verified?
• Monitoring. Could operators detect and interrupt unexpected behavior in real time?
• Human operations. Were responsibilities, escalation paths, and kill procedures clear?
• Supply chain. Did a cloud, evaluation, data, or tooling partner invalidate assumed controls?

The structure is explicit about its purpose: to prevent any investigation from being reduced to the single phrase "model performance." When an agent causes harm, each of those eight layers represents a question that must be answered before the analysis is complete.

Agents create a different reconstruction problem

Security logging has always had a straightforward goal: preserve a trail of what happened so you can understand it later. For most software, that trail looked something like this. A request arrived, a resource was accessed, an outcome occurred. The actor behind the consequential operation was ultimately a person or a service acting on explicit instructions from a person.

Agents change that assumption in a specific way.

When an agent takes a consequential action, the sequence that produced it may be longer and less legible than a single API call or command. The agent may have interpreted an objective, evaluated options, selected a tool, constructed a request, received a response, updated its internal plan, and then acted. The final action is the visible artifact. Everything before it explains why that was the action the agent chose, and whether it was the action anyone intended.

I have written here about how the OpenAI agent involved in the Hugging Face incident was not acting against its instructions but pursuing its objective with the capabilities it had available. I have written about how the Replit incident showed how development and production boundaries can determine the consequence of an agent action. I have written about how the Meta evaluation incident showed that instructions and technical boundaries are not the same thing.

In each case, the surface record alone would not have been sufficient. You would have needed to understand the objective, the identity, the tools, the permissions, and the sequence of decisions. That is precisely the evidence SAFE proposes preserving.

A record is not a gate

The SAFE RFC makes a point that I think practitioners can underweight: "Learning is separate from enforcement. Confidential review should encourage candid reporting, while regulators and affected parties retain their legal rights."

That separation cuts both ways. It means that what SAFE learns from an incident does not become regulatory enforcement. It also means that SAFE's learning process is distinct from the question of whether an action should have occurred in the first place.

Evidence after an action is not a control over whether the action happens. Flight data recorders exist because investigations are valuable, not because recording an approach vector prevents an accident. The record and the control solve different problems.

The SAFE proposal addresses accountability and ecosystem learning: after an agent causes harm or nearly does, SAFE would create a confidential channel to understand what the evidence shows, identify patterns across the industry, and produce recommendations that reduce risk for future deployments. That is genuinely valuable. An ecosystem with shared visibility into agent failures will learn faster than one where each organization investigates in isolation, if it investigates at all.

But accountability after an incident is a different capability than deciding whether a particular action should execute at all. One answers: what happened? The other answers: should this happen?

Both matter. The distinction is worth keeping clear, because conflating them produces a specific failure mode. Organizations that invest in post-incident evidence may believe they have sufficient controls. They do not. The evidence tells you what went wrong after it has already gone wrong. The control determines whether it goes wrong in the first place.

What the evidence list signals

I think the more interesting observation about SAFE is not what it proposes but what it assumes.

Whenever a standard-setting effort produces an evidence framework, it is making a claim about what a reasonable investigator would need to reconstruct what happened. Aviation requires specific flight data because investigators have determined that data is necessary to understand a mishap. Medical device reporting requires specific documentation because regulators have determined those records are necessary to evaluate device-related harm. The evidence framework reflects an investigation theory.

To me, SAFE is making a similar claim about AI agents: that understanding an agent incident requires the prompts, the traces, the tool calls, the identities, the permissions, and the credentials in play at execution time. The RFC puts it plainly: "Trust is not a control; shared evidence and verifiable improvement are how trust is earned."

That list represents an emerging theory of what agent accountability requires. It is not yet a standard. The proposal has no implementation mechanism and no formal safe-harbor. But the fact that an alliance of more than 120 technology organizations, including some of the most significant infrastructure and security companies in the industry, has converged on that list as the minimum evidence for meaningful incident analysis tells you something about how seriously agent behavior is being taken as an operational risk.

We are beginning to define, at the industry level, what it means to know what an agent did.

That is a newer problem than it sounds. A large part of existing security infrastructure was designed around actors who were ultimately human or explicitly instructed by humans. An agent operating with a degree of autonomy, across a sequence of decisions, against resources it was broadly authorized to reach, does not fit neatly into that model. The evidence requirements are different. The reconstruction problem is different. SAFE's evidence list is one of the first serious industry attempts to say: here is what you would need.

Questions worth asking about your agent deployment

SAFE's evidence framework is useful as a diagnostic regardless of whether any organization adopts it. Think of each item in the evidence list not as a compliance requirement but as a question about your current visibility.

If your agent takes a consequential action tomorrow, can you establish what objective it was pursuing at the time?

Can you establish whose identity it operated under, and what that identity was authorized to do?

Can you establish which tools it invoked and in what sequence?

Can you establish what permissions and credentials were available at the moment of the consequential action?

Can you establish what human approval or intervention events, if any, existed during the sequence?

Can you distinguish between what the agent proposed doing and what it actually executed?

If the honest answer to any of those is no, you have a gap in your evidence layer that SAFE is proposing to formalize as a reportable absence. But more practically, you have a gap in your ability to understand what your agent is doing in your own environment, before anything goes wrong.

A bill now asks NIST to write some of this into a federal standard

On 3 September 2026, Representatives Josh Gottheimer and Mike Lawler introduced the Stop Rogue AI Act in the House of Representatives, a bipartisan bill responding directly to the kind of agent incidents this piece already covers, including the OpenAI Hugging Face breach. Moona Intelligence verifies what the bill actually establishes, and what it does not yet establish, as its own dedicated record, connected forward from here rather than restated in full. What belongs in this piece specifically is the shape of the move, not the bill's own detail.

Read against the evidence layer this piece already describes, the Stop Rogue AI Act is a proposal to move part of SAFE's voluntary, post incident evidence list toward something closer to a continuously maintained, federally standardized baseline, backed by procurement leverage over new federal contractors rather than by statute across the whole economy. That is a real escalation in institutional seriousness. Bipartisan legislation naming continuous agent action verification, tamper resistant logging and machine readable agent inventories as the subject of a future federal standard is a different order of attention than an industry alliance's request for comments.

None of this makes the proposal unimportant. A federal standards body spending a year defining what continuous agent action verification and tamper resistant logging actually mean, with procurement leverage behind adoption, is a meaningfully different evidence trajectory than a voluntary industry RFC. The record and the control remain two different problems, whether the record is kept by a 120 member industry alliance or a federal standards body.

The harder question underneath the flight recorder

The flight recorder framing is useful. What it slightly obscures is that the question is not just whether you have a recorder. It is whether what you would find on it would actually explain what happened.

Aviation built its evidence conventions over decades of accident investigation. Investigators knew what data they needed because they had spent years reconstructing failures with insufficient information. The conventions for what a flight recorder must capture reflect accumulated experience about what matters for understanding a mishap.

We are at the beginning of that process for AI agents. To me, SAFE looks like one of the clearest early industry attempts to say: here is what evidence an investigation would need. Here is what a meaningful record looks like for an autonomous system. Here is what you would have to preserve to reconstruct the sequence that produced a consequential action.

If an agent makes a consequential decision tomorrow morning, what evidence will you actually have by the time you need to understand what happened?

A concrete, evaluated construction of the same evidence list

SAFE names the evidence an investigation would need. The Stop Rogue AI Act directs a federal standards body to spend a year defining what tamper resistant logging actually means. Neither, on the material available to this record, specifies a schema, a mechanism, or a measured cost. Agent Flight Recorder, a paper by Laurent Bindschaedler, Quentin Botha, and Christoph Siebenbrunner submitted to arXiv on 1 September 2026, is the first artifact this record has found that does all three for a materially overlapping evidence list, and evaluates the result rather than only proposing it.

This session's own network egress policy blocks arxiv.org and every mirror this record attempted, export.arxiv.org, ar5iv, and Hugging Face's paper index, on every route. What follows rests on repeated, independently phrased web search passes that converged on consistent wording for the paper's identity, its schema, its stated limitations and its evaluation numbers, held at the manual review evidence level rather than treated as a direct read of the primary text. An editor with unblocked network access should read the paper directly before treating any number below as more settled than that.

The paper's own event schema binds eight fields into one record for each action an agent takes: intent, the tool or action proposed and its structured arguments; policy evaluation, which guardrails were evaluated and the resulting verdict together with a policy version; human approval, whether one occurred, the approver's identity, the approval prompt shown, and a timestamp; execution, the exact command, API call, or mutation actually dispatched to the tool runtime, with its concrete parameters; effects, the observed return value, state diff, external response, or error; context provenance, the retrieval sources consumed with their content hashes, upstream tool outputs referenced, and the execution environment's identity; code provenance, the agent binary or version hash and tool wrapper hashes; and delegation provenance, the requesting agent's identity, the parent event's hash, and the scope of authority delegated to it. Naming these as eight separate fields, rather than one undifferentiated log line, is itself the useful move: it keeps what was proposed, what a policy engine decided, what a human approved, what actually ran, and what happened as a result, each independently checkable against the others instead of collapsed into a single narrative an investigator has to take on faith.

Hash chaining links each event to the one before it, and periodic Merkle batching lets a verifier check that one disclosed event belongs to a batch without needing the whole batch. Reported reproducible results tie a cost to that mechanism: the full integrity stack is reported to detect edit, delete, reorder, and fork tampering at 100 percent with zero false positives in the paper's own tamper injection experiments, at a reported median overhead near 48 microseconds per event and roughly 512 bytes of storage per event.

The paper's own architecture keeps two things separate that a shorter summary tends to collapse: the event record, a fixed piece of structural metadata plus a hash of the event's content, and the event payload, the full structured content itself. Only batch roots, not event content, are reported to be committed on chain. The paper reports deploying an anchoring contract to Base Sepolia and measuring gas cost over 100 anchor transactions, with each anchor storing a 32 byte epoch root and a back pointer, and reports a trust model analysis comparing five candidate anchoring mechanisms, including plain signed digests and transparency logs, to identify when the added cost of an on chain commitment buys something a cheaper mechanism does not: a way for two organizations with no shared trusted intermediary to check the same record independently.

That separation carries a consequence this record already keeps in view for every evidence architecture it evaluates: an on chain commitment is a compact promise that some batch of events existed in some order at some time. It is not the evidence itself. Checking a specific disclosed event against that promise still requires the actual payload and a Merkle inclusion proof, both of which have to be stored and retrieved by some other means entirely. A public anchor existing is not the same fact as the underlying record still being available to whoever needs to check it.

What the paper states its own construction does not establish

What makes this artifact worth reading closely, more than the specific numbers, is how precisely it states its own boundary. Three limits stated in the material available to this record are worth keeping exact, because the vocabulary a tamper evidence architecture uses, correctly, invites a stronger reading than the mechanism supports.

First, completeness is explicitly not a goal of the design. The paper's stated guarantee is the integrity of events that were actually recorded, not that every consequential action reached the recorder. If an action bypasses the instrumented gateway the mechanism sits in front of, the recorder never sees it, and nothing in the hash chain or the on chain anchor can tell an investigator that something is missing. Reported detail on this point is specific: suppressing an event inside one epoch is stated to be undetectable from the log alone, because the epoch chain's own back pointers only reveal a missing epoch, not a missing event inside one that is otherwise intact. A record with a perfect, unbroken hash chain and a Base Sepolia anchor proves that what is in the record has not been altered since it was written. It does not prove that everything that happened was written.

Second, the paper's own threat model is stated to stop at the point of full host compromise: after that point, the construction cannot vouch for the truth of newly generated entries, only preserve the integrity of the pre compromise history under its own stated assumptions. A cryptographic anchor is a claim about what a chain of records looked like up to a point in time. It is not a claim that whatever generated a record after an attacker owns the machine producing it was telling the truth.

Third, the approval field's own binding is reported as conditional rather than universal: the material available to this record describes the approval slot as carrying a cryptographic signature or token binding the approver's identity to the specific action only in production deployments of the construction, not as an unconditional property of the schema itself. Read against this record's own recurring distinction between an approval and a gesture, a trace that carries an approval field is evidence that an approval event was recorded. Whether that approval was bound, in a way a third party can verify, to the exact action that then executed is a separate question the schema makes room to answer, not one the mere presence of the field answers by itself. The same discipline applies to the policy evaluation field: a recorded verdict of allow is evidence of what a policy engine decided, not, by itself, evidence that the policy engine held legitimate authority to decide it. And an effect field showing a successful return is evidence of what the tool runtime reported, not independent confirmation that the intended real world effect actually followed, or that whatever executed was the action anyone authorized.

The paper's own evaluation is presented, in the material available to this record, with comparable restraint. Most of the reported workloads are synthetic, generated to exercise the ablation configurations at scale, and the forensic query drills are described as experimenter defined rather than drawn from real investigations. The comparison baseline is reported as plain JSON logging rather than a mature audit platform. Real agent behavior is validated separately: the paper reports replaying five SWE agent trajectories from SWE bench tasks, 38 events in total, roughly split between shell commands, file edits, and file reads, through the same construction, which the paper itself frames as validating that the performance pattern holds on genuine agent traces, not as a claim that the forensic query experiments themselves were run against real investigations. None of that should be read past what it establishes: a working, measured construction for one part of the evidence problem, exercised mostly under conditions its own authors built to exercise it.

Put together with the rest of this record, Agent Flight Recorder is best read as the clearest evidence yet that the flight recorder metaphor can be built, evaluated, and cost, not as evidence that building it closes the gap this record has argued from the start. A tamper evident, on chain anchored record of what an agent did is a genuinely better evidence layer than a server log. It still answers what happened, at the cost this construction reports for answering it well. It does not, on the material available here, answer whether the action should have happened, whether everything that happened was recorded, or whether the specific approval an investigator finds inside it was ever bound to the action that investigator is looking at.

A second paper names the same separation from a different starting point

A paper by Arslan Bromme, submitted to arXiv on 3 September 2026 as A Black Box for Agentic Processes, proposes a closely related architecture from a different starting point: governance, risk and compliance audit readiness under the EU AI Act, NIS2 and the Cyber Resilience Act, rather than a measured forensic construction. This session's own network egress policy blocked arxiv.org and every mirror attempted on every route, so what follows again rests on repeated, independently phrased web search passes held at the manual review evidence level rather than a direct read of the primary text.

The paper's own reported pipeline names six stages: capture, canonicalize, hash, anchor, certify, verify. It treats canonicalization, converting a captured event into a stable representation such as normalized JSON, as a security relevant step in its own right, since an agentic event may arrive as a JSON object, a log record, a transcript, a screenshot, a tool call payload or an exported audit artifact, and a verifier can only recompute a matching hash if the canonical form is stable. Like Agent Flight Recorder, only a digest or an aggregate commitment is reported to reach the chain; the communications, approvals, tool calls and artifacts themselves are not.

Where this paper earns a place in this record is its own evidence model, stated directly rather than left implicit. It reports separating what a digest verification actually establishes, that a retained artifact matches a prior commitment, from four further properties it treats as distinct: whether the capture mechanism itself faithfully observed the underlying event, whether every relevant event was captured within a defined scope, whether two anchored events occurred in the causal order their anchoring or timestamp order happens to suggest, and whether whoever submitted an anchor was authorized to. The paper is reported to state plainly that the architecture does not prevent agent misbehaviour, does not establish the semantic truth of a captured record, and presents no empirical performance or security evaluation of its own, a materially different evidence posture from Agent Flight Recorder's own measured construction covered above.

The temporal ordering point is worth holding onto on its own. Two events anchored to a public ledger in a given order can reflect batching, network propagation, confirmation timing, submission timing or the anchoring service's own behaviour, none of which is the same fact as which action in a workflow caused or authorized which. An investigator reading two anchored records in sequence still needs independent dependency evidence before treating that sequence as the causal story of what happened, exactly as this record's own recurring distinction between a record and a gate already requires for authorization.

Put together with Agent Flight Recorder, the pattern across both artifacts is now two independently authored proposals converging on the same underlying claim from different directions: a cryptographically anchored, tamper evident record of agent activity is a genuinely better evidentiary substrate than a plain log, and neither paper, on the material available to this record, claims that substrate closes the gap between what was recorded and what happened, who was authorized, or why one event followed another.