Vibe-Trading's Audit Gate Returned PASS on a Report Where Zero Data Points Were Verified

A gate that says PASS is supposed to mean something was checked and nothing failed. Vibe-Trading's own research report audit tool, report_audit, exists precisely to stand between a research report's own numeric claims and publication, sampling reported data points and comparing each against an independently fetched value. For as long as its verdict function existed before 6 September 2026, that gate could return its most confident word, PASS, having checked absolutely nothing.

What pull request 1362 actually fixed

Pull request 1362, titled fix(report-audit): fail closed when nothing was verified, was opened 5 September 2026 by contributor cgycorey and merged 6 September 2026 as merge commit cc4774ed42d9f2e24e66ea4a7ee8d283c553bbba, with parent commit 9c063312. Independently cloning the public HKUDS/vibe-trading repository and reading both commits directly, rather than relying on the pull request's own rendered description, confirms the mechanism precisely. The affected function, render_verdict in agent/src/tools/report_audit_tool.py, iterates a list of per point results, each carrying a reported_value and an optional fetched_value, and is meant to produce a PASS or FAIL verdict for the sampled report. Before the fix, a point whose fetched_value was None was skipped with a bare continue, contributing to neither the running total nor the fail count. At the end of the loop, the function set verdict to PASS whenever fail_count, the length of the accumulated list of failed points, was zero, with no separate check on whether total, the count of points actually evaluated, was itself zero. An evidence set containing only skipped points therefore produces fail_count 0 by construction, and the function, before the fix, reported that as a pass.

Three changes in one commit, not one

The fix is worth separating into its distinct parts, because each closes a different way the same aggregate rule could be gamed or could simply misbehave. The first, and the one this record leads with, is the explicit zero total check: after the main loop, if total equals 0, the function now appends a synthetic failure item with the reason quoted above, so the returned verdict is FAIL rather than PASS whenever nothing was actually checked. The second addresses what happens when a fetched value is present but unusable. Before the fix, a non numeric fetched_value, a string such as "about 100", NaN, or an overflowing value, reached a bare float conversion further down the same function and raised an uncaught exception, a crash rather than a verdict. Reproduced directly for this record, calling the function from before the fix with a single point whose fetched_value is the string "about 100" raises a value error refusing to convert that string to a float; the identical call against the fixed commit returns a clean FAIL, with the point's own reason stating the fetched value is not a finite number. This distinction matters on its own terms: a present but garbage fetch is not the same fact as an absent one, and the fixed function is explicit in its own comments that treating the two as equivalent would let an agent, honest or not, launder a hard to verify number out of the audit by padding it with junk rather than a real fetch. The third change makes a non finite second source, fetched_value2, degrade to a single source verdict rather than raise the identical class of exception the code before the fix carried for the primary fetched value.

What stayed the same, verified directly

None of the three changes touches the one behavior this record is careful not to overstate as fixed, because it was never broken in the sense of producing a wrong result: a genuinely absent fetched_value, one the agent never attempted to fetch or could not obtain, remains legitimately skippable, both before and after the fix. Reproduced directly for this record against both commits, a call with one point carrying fetched_value None and a second point carrying a fetched_value equal to its reported value returns PASS at both commits, with total counting only the one point that was actually checked. Partial coverage and zero coverage are evidenced here as two distinct states, and the fix touches only the second.

Five regression tests, run directly against both commits

Pull request 1362 adds five new test functions to agent/tests/test_report_audit_tool.py: test_verdict_zero_verified_points_fails_closed, test_verdict_garbage_fetched_value_fails_point, test_verdict_garbage_fetch_cannot_pad_a_clean_report, test_verdict_absent_fetched_with_good_point_still_passes and test_verdict_garbage_second_source_falls_back_to_single. Reading each directly against the diff confirms they assert exactly the properties described above: a call with two missing fetch points returns FAIL with total 0; a call with one garbage fetch returns FAIL citing a non finite value; a garbage point cannot be diluted by a clean sibling point into an overall PASS; a genuinely absent fetch alongside a passing point still returns PASS; and a garbage second source degrades to a single source PASS rather than crashing. This record's own independent, isolated execution of render_verdict's literal source at both commits, reported above, reproduces the same pattern, failing at the commit before the fix and passing at the fixed commit, that these tests assert, without relying on the pull request's own account of what they show.

What remains unestablished

This record does not extend beyond what the evidence supports. Nothing available to it states that report_audit's own zero evidence PASS was ever exercised against a report that actually reached publication or a reader; the reproduction here runs the gate function directly, against synthetic inputs, not against a live agent run. Nothing states whether report_audit is the only gate standing between a drafted report and publication in Vibe-Trading's own pipeline, or whether a separate, later step would also have caught an unverified report before it shipped. Current main, independently diffed against the fixed commit for this record, carries no further change to render_verdict, so the fix has not since been weakened or reverted. And, as stated above, this record does not treat the fix as having established that a fetched value is true; it has established only that an empty checked set cannot pass as though it were a checked and passing one.

The Authority Provenance ledger

Principal and mandate. Whoever operates Vibe-Trading's own research pipeline and relies on report_audit's verdict as the gate between a drafted report and publication. Unnamed in the pull request itself; this record treats the principal as whoever configures that pipeline, consistent with how the tool's own description, quoted from its source, states its purpose: auditing a report's numeric data points for accuracy before publishing.

Submitted action. A verdict call carrying a list of sampled data points, each with a reported value and, where an upstream fetch succeeded, a fetched value, submitted by the agent's own tool calling loop after the extract phase has sampled a report and separate tool calls have attempted to fetch each sampled point's authoritative value.

Evidence bound parameter. Each point's own fetched_value, and, distinctly, whether that value is present at all, present but unusable, or genuinely absent. Before the fix, the function conflated the second and third states with the outcome of the first only at the aggregate level, by never requiring a non empty checked set for a positive verdict.

Normalized evidence object. The accumulated total, pass_count and fail_count the loop builds. Before the fix, a call in which every point's fetched_value was None produced total 0, pass_count 0 and fail_count 0, an object indistinguishable, at the point the verdict is computed, from one where every point had been genuinely checked and none had failed.

Authority presented. The gate's own verdict, PASS, computed from a rule that read only whether fail_count equalled 0, presented as though it certified the report's sampled numeric claims.

Authority required. A PASS verdict requires that at least one sampled point was actually checked against a fetched value and found to agree; an aggregate rule that a positive verdict follows automatically from an empty checked set never held the authority a PASS verdict is meant to carry.

Decision. Automated, both before and after the fix: render_verdict's own aggregate rule, with no human review step inside the gate itself. What changed on 6 September 2026 was whether an empty checked set could reach the same decision a genuinely checked and passing set reaches, not whether a human was in the loop.

Where this sits against what Moona Intelligence has already written

Vibe-Trading's own MCP order gate priced a buy limit order at the market quote alone, letting a mandate cap an order's checked cost below what its own submitted terms permitted, fixed five days after an identical gap on the framework's direct SDK trading path. This record concerns the same repository and, loosely, the same desk level theme of a gate that computes a number from an incomplete basis, but a mechanically distinct failure: that record's gate always evaluated a real, non empty set of parameters and evaluated them against the wrong basis value; this record's gate evaluated an evidence set that could be entirely empty and let an aggregate counting rule, fail_count equal to 0, treat that emptiness as identical to a genuine pass. Nothing here is folded into that record's own AEW-034 or AEW-035, and nothing there is folded into this one.

Claimed authorization accepted without verification describes an agent that treats a bare assertion of authority as though it were a verified mandate. This record's gap is adjacent but distinct: report_audit's own verdict function does attempt verification, fetching values and comparing them, and the failure is not that an assertion went unchecked but that the function's own aggregate logic collapsed a checked set of size zero into the identical outcome a checked and passing set produces. The lesson generalizes from a different angle: not every unverified claim announces itself as a claim; some arrive disguised as the verdict of a verification process that ran, and produced nothing, on nothing.