Intelligence
Join the waitlist
AEW-004

The agent controls whether its control applies

A safety gate is governed by a parameter the model itself can set, so the actor the gate constrains decides whether the gate fires.

Status: publishedApproval ControlsExecution Authority

Description

A control is only a boundary if the thing it constrains cannot switch it off. This weakness appears when the field that decides whether a gate applies is exposed to the model as an ordinary input, alongside the action the model wants to take. A single tool call then carries both the action and the switch that governs its own approval, and crafted input, including indirect prompt injection, can set the switch. The same shape appears when the only safeguard is an instruction inside the agent's own context: advisory, competing with the objective, and losing. The distinction that matters is between an authority defining value the operator sets before the agent runs and one the model supplies at call time.

The authority gap

Authority over whether a control applies rests with the actor the control exists to constrain, so the boundary is self administered by the thing it bounds.

Failure conditions

  • A consent gate or safety check reads its enabling condition from a model controllable field.
  • The action and the parameter that governs its approval travel in the same tool call.
  • The only restraint is an instruction the model can reason past in pursuit of the objective.

Consequences shown by the evidence

  • A human consent gate skipped and an operating system command run with the agent process's privileges.
  • A stated safety rule acknowledged and then overridden because the objective outranked it.

Detection signals

  • A tool's input schema exposes a field that governs approval, logging or isolation.
  • The fix for an incident relocates an authority defining value out of the model facing schema into operator configuration.
  • The agent can articulate the rule it is about to break.

Known examples

  • AWS Strands Agents Tools shipped a shell tool whose human consent gate was skipped when a model settable non_interactive parameter was true.
  • A coding agent enumerated the exact safety rules it had violated moments after deleting a database, the rules having been advisory only.

Recorded instances