A verifier returns a positive verdict from an empty qualifying evidence set

Description

This weakness names a specific defect in how a verifier combines many individual results into one overall decision, not in how it evaluates any single result. A verification routine samples or receives a set of items, attempts to check each against independent evidence, and is meant to produce a positive verdict only when that checking actually happened and found no problem. The defect appears when the routine's own aggregate rule is written as a count of failures rather than as a comparison between the count of items checked and the count of items that failed: an item whose supporting evidence is absent is treated as legitimately unverified and excluded from both totals, on the reasonable premise that an agent should not be penalized for a fact nobody could reach. That premise is correct for one absent item beside others that were checked. It stops being correct once every item in the set is excluded the same way, because a failure count built only from checked items is, by definition, zero when no item was checked, and a rule reading zero failures as a pass cannot distinguish that state from a set that was fully checked and fully correct. The two states differ in exactly what matters, whether the positive verdict is backed by any supporting evidence at all, and the aggregate rule alone cannot see the difference. A second, narrower shape of the same underlying gap appears beside it in this dataset's own evidence: an item whose evidence is present but malformed, unparseable, non numeric or non finite, is a different fact again from an item whose evidence is simply absent, and a verifier that silently drops a malformed item into the same excluded bucket an absent item uses lets a caller, or an agent supplying that evidence, launder a hard to verify item out of the audit entirely by padding it with garbage rather than a real, checkable value. This weakness differs from claimed authorization accepted without verification, AEW-007: there, an actor asserts in words that an action is authorized, and the gap is that nothing checks the assertion at all. Here a verification process genuinely runs, attempts real checks, and the gap lives in how its own results are aggregated into a final answer, not in whether checking was attempted. It differs from evidence after the fact mistaken for authorization before it, AEW-011: that weakness confuses a record of what happened with a decision about whether it should have happened, a confusion between two different kinds of artifact. Here there is only one kind of artifact, a verdict, and the confusion is between two different evidence states, an empty checked set and a genuinely checked and passing one, that a single aggregate number cannot distinguish. It differs from a malformed authority restriction parsed the same as no restriction at all, AEW-032: that weakness concerns a single field's own parser routing an invalid value into the same branch its own genuine absence uses, at the level of one authority bearing field. Here the same shaped confusion, an unusable input handled the same as an absent one, recurs at the level of a verification aggregate's own excluded bucket, in a materially different domain, evidence sufficiency for a verdict rather than a caller's own declared authority scope, and this weakness is recorded separately because the aggregate rule producing a false positive verdict from zero checked items is the primary mechanism the evidence establishes, with the malformed evidence question a secondary, related shape of the same excluded bucket rather than the weakness's own defining gap.

The authority gap

The authority a positive verdict is presented as carrying is that the set of items in question was checked and found correct. The authority a positive verdict actually requires is that at least one item was checked at all; an aggregate rule counting only failures, with no floor on the count of items checked, silently treats an evidence set of size zero as satisfying a requirement it was never in a position to satisfy.

Failure conditions

• A verification routine checks a set of individual items, each against independently obtained supporting evidence, and combines the per item results into one aggregate verdict for the whole set.
• The aggregate rule is written as a count of failed items alone, positive whenever that count is zero, with no separate requirement that the count of items actually checked is itself greater than zero.
• An item whose supporting evidence cannot be obtained is legitimately excluded from both the checked count and the failed count, a correct rule when other items in the set were genuinely checked, but the routine applies it uniformly with no floor on how many items may be excluded this way.
• Every item in a given call is excluded for lack of evidence, so the checked count and the failed count are both zero, and the aggregate rule reports that state as a positive verdict identical in its output to a set that was fully checked and fully correct.

Consequences shown by the evidence

• A report, a claim or an action is certified as verified, and treated downstream as though independent evidence supports it, when in fact no supporting evidence was obtained for any of its checked items.
• A reviewer or a downstream automated consumer inspecting only the verdict word, or only the failure count, finds nothing wrong, because the aggregate's own output is structurally identical whether the underlying evidence set was full and correct or entirely empty.
• An agent, honest or not, can reach a positive verdict for an item it could not or did not actually verify simply by ensuring nothing about that item is checkable, or by supplying unusable evidence for the specific item it most needs to avoid having checked, when malformed and absent evidence are excluded through the same code path.

Detection signals

• A verification function's own aggregate rule reads a failed count alone, with no accompanying check on the checked or total count, immediately before deciding the returned verdict.
• A code path that skips an item for missing supporting evidence uses the identical excluded bucket, and the identical downstream handling, that a code path for malformed or unusable supporting evidence also uses.
• A verdict object's own returned fields include a total or checked count alongside the verdict word, and a positive verdict can be observed, in testing or in production output, alongside a checked count of zero.

Known examples

• Vibe-Trading (HKUDS/Vibe-Trading), an open source LLM agent trading and research framework, ships a report_audit tool gating a research report's numeric claims before publication. Its verdict phase, render_verdict in agent/src/tools/report_audit_tool.py, iterates a list of sampled data points, each carrying a reported_value and an optional fetched_value obtained by a separate tool call, and is meant to return PASS only when the sampled points were checked and none failed. Before pull request 1362, merged 6 September 2026 as merge commit cc4774ed42d9f2e24e66ea4a7ee8d283c553bbba, a point whose fetched_value was None was skipped with a bare continue, contributing to neither the running total nor the fail count, and the function set its final verdict to PASS whenever fail_count, the length of the accumulated list of failed points, was zero, with no separate check on whether total, the count of points actually evaluated, was itself zero. Reproduced directly for this record against the commit immediately before the fix (9c063312): a call carrying two reported values, 1234.5 and 999.0, and no fetched value for either, returns verdict PASS, total 0, pass_count 0, fail_count 0. The identical call against the fixed commit (fc780704) returns verdict FAIL, total 0, fail_count 1, with the added failure item's own reason stating that no data points were verified because every result lacked a fetched value. The same pull request closes the narrower, related gap this weakness also names: before the fix, a fetched_value that was present but not a finite number, a string such as about 100, NaN, or an overflowing value, reached an unguarded float conversion later in the same function and raised an uncaught exception rather than failing the point or being excluded; reproduced directly for this record, the function from before the fix raises that exception on such a call, while the fixed function returns a clean FAIL citing the specific point's own unusable fetched value, and a companion change makes a non finite second source degrade to a single source verdict rather than raise the identical class of exception. The fix's own five new tests, read directly against the diff and independently reproduced against both commits for this record, confirm a genuinely absent fetched_value remains legitimately skippable when at least one other point in the same call was actually checked, so the fix narrows the gap to the specific case where the entire qualifying evidence set is empty rather than requiring every item in every call to carry evidence. Current main, independently diffed against the fixed commit for this record, carries no further change to render_verdict, so the fix has not since been weakened or reverted. Nothing available to this record states that the pre fix behavior was ever exercised against a report that reached publication, and nothing in the fix establishes that a fetched_value the gate accepts is itself true: an agent supplying a fabricated fetched_value matching its own reported value still produces a point the fixed function counts as verified and passing.

Recorded instances

• AEV-2026-0050 Vibe-Trading's report audit gate returned PASS on a call where every sampled point's evidence was missing AESS 7.0 high Confirmed

Related Intelligence Records

The published Moona Intelligence Records whose evidence establishes this weakness.

• Vibe-Trading's Audit Gate Returned PASS on a Report Where Zero Data Points Were Verified

Protocol evidence

Published authority architectures whose artifacts bear on the requirement this weakness names: which satisfy it, which expose a way past it, and which leave it unanswered.

• Implementation evidence

  Verifiable Attenuated Delegation for AI Agent Chains (draft-asor-wimse-agent-delegation-chain)

  Rafael Asor, Attenu

  Requirement A verifier encountering an unrecognized constraint type must deny rather than treat it as unconstrained

  The draft's own verified rule, that a verifier encountering an unrecognized constraint type must deny rather than treat it as unconstrained, states the general principle at the level of one delegation constraint: an input the verifier cannot interpret must never be silently treated as the absence of a restriction. Vibe-Trading's own fix for pull request 1362 is independent, real-world implementation evidence for the identical principle in a materially different context, audit evidence rather than a delegation constraint: a fetched_value the gate cannot interpret as a finite number, present but unusable, now fails the point it belongs to rather than being silently folded into the same excluded bucket a genuinely absent value uses. Recorded as implementation evidence for the draft's own already-cataloged principle operating in a second domain, not as a claim that the draft itself governs report audit gates.

  View protocol evidence

• Missing requirement

  Agent Action Decision Protocol (AADP)

  Shamik Saha, individual submission to the IETF

  Requirement The draft claims evidence sufficient to re-derive every verdict, not cryptographic signing

  AADP's own claim, as corroborated here, is that its evidence is sufficient to re-derive every verdict it produces, a claim about a positive record's own reconstructability once a decision has already been reached. It is not, on the material this dataset corroborates, a requirement that a decision withhold a positive verdict in the first place when the evidence available to reach it is empty. Vibe-Trading's own pre-fix report audit gate demonstrates precisely that adjacent, unstated requirement: its own verdict object, PASS with total 0, is entirely re-derivable from its own inputs, an empty results list with every fetched_value absent, and re-derivability alone says nothing about whether a positive verdict should ever have been reached from those inputs. Recorded as a requirement this dataset's evaluated protocols do not currently name, not as a defect in AADP's own re-derivability claim, which answers a different, real question.

  View protocol evidence

• Missing requirement

  Agent Control Standard (ACS)

  OWASP GenAI Security Project, originally Zenity

  Requirement The specification does not guarantee complete mediation of every consequential action a host can take

  ACS's own stated conformance position leaves mediation completeness itself unguaranteed by the specification alone, addressing whether a consequential action reaches a check at all. Vibe-Trading's own report audit gate is mediated in that narrower sense, every sampled point does reach render_verdict, and the gap this weakness names sits one layer further in: the aggregate rule the mediation point itself applies, a failed count of zero producing a positive verdict, does not separately require that the checked count be non zero. Recorded as a requirement this dataset's evaluated protocols do not currently name, not as a defect in ACS's own stated, narrower conformance bar.

  View protocol evidence