Skip to content

fix(agent): mid-turn compact gate fires on the server meter, not just the local estimate - #236

Merged
justrach merged 1 commit into
mainfrom
fix/midturn-compact-server-meter
Jul 16, 2026
Merged

fix(agent): mid-turn compact gate fires on the server meter, not just the local estimate#236
justrach merged 1 commit into
mainfrom
fix/midturn-compact-server-meter

Conversation

@justrach

Copy link
Copy Markdown
Owner

Problem

A long-running codex/.responses turn died mid-turn: after a tool call returned, the next request was rejected with "Your input exceeds the context window." graff auto-recovered (compact ~511k → retry → summary), but the #193 pre-send gate should have compacted before shipping the over-cap request. It didn't.

Root cause — two gates, two different signals

  • Between-turns gate (mainloop.zig:683/734) trusts last_context_tokens, the accurate server-reported count — but only runs after a turn finishes.
  • Mid-turn pre-send gate (inputOverCompactThreshold) trusted only the local fullInputEstimateTokens byte/4 estimate, and runs every loop iteration.

The local estimate structurally under-counts codex histories: retained encrypted-reasoning items are billed large server-side but are compact base64 handles locally. So during a long single turn the local estimate stayed under the threshold while the server had already reported ~511122 via last_context_tokens (refreshed mid-turn by recordUsageResponses) — and the mid-turn gate simply never looked at that number. It had the right answer in hand and ignored it.

Fix

Make the mid-turn gate also fire on the server meter — the exact signal the between-turns gates already trust:

if (self.last_context_tokens >= threshold) return true;

Plus two supporting changes:

  • Test correctness: the #193 test builds Agent = undefined, so the new last_context_tokens read was uninitialized-memory UB that would deterministically break the suite. Explicitly zero it, and add a meter-only case that trips the gate with an empty local history.
  • Hardening: fullInputEstimateTokens now falls back to the bytes counted so far on a serialize failure (catch return d.fullCount()/4) instead of a spurious 0, so a failed serialize can't blind the pre-send gate and the #202 meter floor at once. (Zero-risk: the Discarding sink is infallible today; this future-proofs a stricter sink.)

Residual gap (unchanged from prior behavior)

A cold-start giant first request of a turn, while last_context_tokens is stale-low from a prior small turn, is still guarded only by the local estimate + the #201/#203 output caps. The reported failure mode (multi-round-trip, meter already at 511k) is now caught.

Verification

  • zig build clean
  • zig build test195/195 unit tests pass (incl. the modified #193 test exercising the new meter path)
  • zig build repl-test19/19 REPL tests pass
  • Diff: +21/−1 in src/agent_request.zig

… the local estimate

The #193 pre-send gate (inputOverCompactThreshold) trusted only the local
byte/4 estimate (fullInputEstimateTokens). On codex/.responses that estimate
structurally under-counts retained encrypted-reasoning items — they're billed
large server-side but are compact base64 handles locally — so on a long single
turn the local estimate stayed under the threshold while the server had already
reported ~511k via last_context_tokens (refreshed mid-turn by
recordUsageResponses). The next resend was then rejected for "input exceeds the
context window" before any compaction ran — the gate had the right answer in
hand and ignored it.

The mid-turn gate now also trips on last_context_tokens — the same accurate
server meter the between-turns gates (mainloop.zig:683/734) already trust:

    if (self.last_context_tokens >= threshold) return true;

Supporting changes:
- The #193 test builds Agent = undefined, so the new last_context_tokens read
  was uninitialized-memory UB; explicitly zero it and add a meter-only case that
  trips the gate with an empty local history.
- fullInputEstimateTokens now falls back to the bytes counted so far on a
  serialize failure (catch return d.fullCount()/4) instead of a spurious 0, so a
  failed serialize can't blind the pre-send gate and the #202 meter floor at once.

zig build clean; 195/195 unit + 19/19 REPL tests green.

Co-authored-by: Codegraff <blackfloofie@codegraff.com>
@justrach
justrach merged commit ed69f29 into main Jul 16, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant