Skip to content

fix(agent): detect a repeated same-tool fan-out - #89

Open
LukasParke wants to merge 23 commits into
mainfrom
lukeparke/doom-loop-fanout-streaks
Open

fix(agent): detect a repeated same-tool fan-out#89
LukasParke wants to merge 23 commits into
mainfrom
lukeparke/doom-loop-fanout-streaks

Conversation

@LukasParke

@LukasParke LukasParke commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

The gap

#73 keys a tool's streak on its last fingerprint (this.tools.get(toolName)), so a fan-out of distinct arguments reissued verbatim never accumulates evidence. Each round's first call has a different fingerprint than the previous round's last call, which resets the streak to 1 before the matching call arrives.

Measured against main before this change:

8 rounds x 3 distinct-arg calls (read a/b/c):  detections = 0
control, 1 call per round (read a):            none, observe, block, block

Zero detections across 24 calls the model had no business making. read(a), read(b), read(c) on repeat is the dominant shape in parallel-tool-calling agents, so this is the common case rather than an edge one.

Worth being precise about what #73 does handle: N identical calls fanned out in one round correctly count once (the duplicateInRound path, B3 in the remediation suite). The miss is specifically distinct arguments across a repeating round.

The fix

A round's identity for one tool is now the set of fingerprints it was called with, compared across rounds — not the last call.

The set is only complete once every call in the round has arrived, so a fan-out scores on the call that completes the match, and the round's earlier calls report the pre-match streak. That is deliberate: a partial fan-out genuinely is not yet a repeat.

r0: none, none, none
r1: none, none, observe     <- set matches r0 on the third call
r2: none, none, block

Ordering within the round does not matter (the set is sorted), a changed member resets the streak, and a strict subset is not a repeat.

Unchanged: single-call round timing, in-round duplicate collapsing, and verdict payloads. The persisted shape gains one additive optional field (roundFingerprints on DoomLoopStreak) so a fan-out's streak survives save/resume paired with the set that earned it; old blobs restore with their old semantics. Resumed single-call streaks behave exactly as before.

Tests

7 new tests in doom-loop-fanout.test.ts:

  • accumulates across repeated fan-out rounds
  • order-insensitive within a round
  • resets when membership changes
  • a strict subset is not a repeat
  • single-call rounds behave exactly as before
  • in-round duplicates still count once
  • a resumed streak still increments after restore()

3 of the 7 fail without the fix — verified by stashing the change and re-running.

Verification

  • vitest run (packages/agent): 753 passed, 61 files — the 115 existing doom-loop tests all still pass
  • tsc --noEmit: clean (needed exactOptionalPropertyTypes care on the new optional fields)
  • biome check: clean
  • Changeset: patch

Context

Found while evaluating whether #73 supersedes the router-side port in openrouter-web#30170. It largely does — the ladder, escalation, text detectors, and in-loop stop are all things the router plugin cannot do from resolveEndpoints. This was the one axis where the router port was stronger, because the same gap was caught there in review and fixed by keying on a round fingerprint. Closing it here so the router can drop its duplicate and depend on the SDK.

Also verified while I was in here, no action needed: the two-tier fail-open in model-result.ts:1344 (raw-args fallback, then skip-detection) is correctly per-call, so one unhashable value cannot zero a whole request's detection.

One thing I did not change, flagging for a decision: the detector ignores tool results, so identical arguments with changing results reach block by round 3 — a polling tool returning running then done looks like a loop. loopKey: false is the intended exemption, but it is opt-out per tool rather than automatic. Folding a result digest into the round identity would make polling self-exempting; happy to follow up if you want that.

API example

For callModel users, doomLoop is configured exactly as before — what changed is when it fires:

import { callModel } from '@openrouter/agent';

const result = callModel(client, {
  model: 'z-ai/glm-5.2',
  input: 'Summarize these files.',
  tools: [readTool],
  // Unchanged config; ladder default is observe@2, block@3, stop@6.
  doomLoop: true,
});

// Model reissues the SAME three-call fan-out every round:
//   round 1: read(a), read(b), read(c)
//   round 2: read(a), read(b), read(c)   <- identical set
//
// was: no detection, ever — each round's first call reset the streak, so a
//      fan-out spun indefinitely while single calls tripped at round 2.
// now: round 2 -> streak 2 (observe), round 3 -> streak 3 (block), and EVERY
//      call of the round is refused at block, so the fan-out stops spending.
//
// A round that ADDS work is progress, and resets to 1:
//   round 3: read(a), read(b), read(c), read(d)   <- no verdict
//
// `loopKey` still runs exactly once per checked call. Persisted state gains
// one additive optional field so fan-out streaks survive save/resume.

API example — new public surface

This PR adds public API: DoomLoopMonitor.declareRound, plus resolveDoomLoopOption and ResolvedDoomLoopConfig exports (the monitor was previously exported without its config resolver, so it could not be constructed from the package at all). Consumer usage:

import { DoomLoopMonitor, resolveDoomLoopOption } from '@openrouter/agent';

// now: constructible from the public entrypoint (was: TS2345 / not exported)
const monitor = new DoomLoopMonitor(resolveDoomLoopOption(true));

for (const [round, batch] of batches.entries()) {
  // NEW: declare the round's complete set BEFORE recording any of its calls.
  // Undeclared multi-call rounds are scored per call and accumulate only on
  // the last-recorded member, order-dependently.
  await monitor.declareRound(
    round,
    batch.map((call) => ({ toolName: call.name, keyMaterial: call.arguments })),
  );
  for (const call of batch) {
    const { verdict } = await monitor.recordToolCall(call.name, call.arguments, round);
    if (verdict?.action === 'block') refuse(call, verdict.message);
  }
}

// State round-trips as plain JSON, so per-turn serverless topologies
// accumulate fan-out evidence across process boundaries.

Note on the implementation, since review

The first commit scored a round's fingerprint set as it accumulated, which
made a superset round transiently match its predecessor — blocking calls that
represented real progress, order-dependently. The engine now declares a round's
complete set before any of its calls is scored (declareRound). Subsequent
commits fixed fallout from that seam: streak sharing is scoped to declared
members, a non-member can no longer clobber the round's set, and loopKey runs
once per checked call. See the review threads for the full trail.


Open in Devin Review

Streaks compared a tool's last fingerprint, so a fan-out of distinct arguments
reissued verbatim never accumulated evidence: `read(a), read(b), read(c)` has a
different last call every round, and each round's first call reset the streak to
1. Measured before this change — 8 identical rounds of a 3-call fan-out produced
zero detections, while single-call rounds tripped at round 2. Distinct-argument
fan-out is the dominant shape in parallel-tool-calling agents, so this was the
common case going unseen.

A round's identity for one tool is now the set of fingerprints it was called
with, compared across rounds. The set completes only once every call has
arrived, so a fan-out scores on the call that completes the match and the
round's earlier calls report the pre-match streak — a partial fan-out is not yet
a repeat.

Unchanged: single-call rounds, in-round duplicate collapsing (one decision per
(tool, fingerprint) per round), resumed streaks, persisted state shape, and
verdict payloads. The new round fields are run-local and never serialized.

Verified: 7 new tests covering accumulation, order-insensitivity within a round,
reset on changed membership, subset-is-not-a-repeat, and the single-call and
resume controls. 3 of them fail without this fix. Full suite 753 pass, typecheck
and biome clean.
Comment thread packages/agent/src/lib/doom-loop.ts Outdated
duplicateInRound = seen.includes(fingerprint);
streak =
previous.priorRoundFingerprints !== undefined &&
setsMatch(previous.priorRoundFingerprints, roundFingerprints)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A round that is a strict superset of the previous round transiently matches it, so a progressing round can be blocked.

The set is compared after every call while the round is still filling, so any round whose prefix (in sorted-merge terms) momentarily equals the prior set scores as a repeat — including rounds that add new work.

Trace with the default ladder (observe 2 / block 3 / stop 6)

r0 = [a,b], r1 = [a,b], r2 = [a,b,c] (the model added a third file — real progress):

r1: a -> streak 1        (set [a]   != prior [a,b])
    b -> streak 2        (set [a,b] == prior [a,b])   observe   ✔ correct
r2: a -> streak 1        (set [a]   != prior [a,b])
    b -> streak 3        (set [a,b] == prior [a,b])   BLOCK     ✘ read(b) is refused
    c -> streak 1        (set [a,b,c] != prior)

checkDoomLoopBeforeExecution turns that verdict into a hook_blocked outcome (model-result.ts, runToolWithHooks), so a legitimate call never executes and the model gets a "you are looping" error instead of its result. On main all three r2 calls score 1 and nothing fires.

It is also order-dependent, which contradicts the PR's "ordering within the round does not matter": emitting r2 as [c,a,b] never produces the transient match and nothing fires. Same shape applies to expanding fan-outs ([a], [a,b], [a,b,c], …), which accumulate streaks they should not.

The tests cover the strict-subset direction ("does not treat a partial repeat as a repeat") but not the superset direction, which is the one that fires.

Options: score a round's identity only once it is known complete (evaluate the previous round at the next round boundary, accepting one round of detection latency), or gate mid-round matches so a set that later grows cannot have already produced a blocking verdict. Either way, please add a superset regression test ([a,b], [a,b], [a,b,c] ⇒ no verdict on b).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed — reproduced all three claims against head (f46f74c), with the default ladder:

[a,b], [a,b], [a,b,c]  => [["none","none"],["none","observe"],["none","block","none"]]
[a,b], [a,b], [c,a,b]  => [["none","none"],["none","observe"],["none","none","none"]]
[a], [a,b], [a,b,c], [a,b,c,d]
                       => [["none"],["observe","none"],["none","observe","none"],["none","none","observe","none"]]

So read(b) is blocked in a round that added new work, the same round emitted in a different order fires nothing, and expanding fan-outs accumulate streaks. The order-dependence is the part I think settles it: the PR claims ordering within a round does not matter, and it demonstrably does. (Devin flagged the same thing and left it as a judgment call; I read the permuted case as making it a bug rather than a defensible trade-off.)

Root cause is that roundFingerprints grows incrementally but is compared against a completed prior set, so any prefix that transiently equals the prior set scores. Retracting after the fact cannot work — the block has already been returned and the call refused.

Not pushing a fix yet: the sound options change semantics the module documents as a cross-port spec, so I have taken the direction to the author rather than picking one myself. Leaving this thread open. The superset and order-permuted regression tests you asked for will come with the fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in eb3b51d — took your first option, adapted to avoid the latency cost.

Rather than deferring evaluation to the next round boundary, the engine now declares a round's complete call set before any of its calls is scored (declareRound, called from all three beginDoomLoopRound sites — they each already had the batch in hand). So the comparison is whole-round against whole-round with no detection latency, and single-call timing is untouched. I avoided boundary scoring specifically because it would shift single-call rounds from none, observe, block, block to none, none, observe, block, breaking the "unchanged: single-call rounds" guarantee for the most common shape.

Verified against your trace — the superset round now scores nothing, and the permuted form is identical:

[a,b], [a,b], [a,b,c]  => [...,["observe","observe"],["none","none","none"]]
[a,b], [a,b], [c,a,b]  => identical

Both regression tests you asked for are in, plus an expanding-fan-out case. All three fail against f46f74c with the buggy values.

Your other four findings are also addressed in the same commit:

  • restore() dead state — confirmed unreachable and removed. priorStreak was only read under isSameRound, which needs previous.round !== undefined, and restore() deliberately leaves round unset. I split the remaining in-round duplicate tracking into a separate seenThisRound field so the two roles are no longer conflated.
  • Stale docs — updated the recordToolCall docstring, the file-header "Round-scoped streaks" bullet (now states the round-set rule and the declare-before-scoring requirement for the Python/Go ports), and DoomLoopVerdict.streak. The verdict message no longer claims "identical arguments" for a fan-out; it says "the same set of arguments" and notes the parallel-call count.
  • Partial mitigation at the block rung — fixed rather than documented. Every call in a repeating round now reports the round's streak, so a blocked fan-out stops spending entirely instead of executing N-1 calls per round. That is the one intentional behavior change beyond the bug fix.

@cortex-github-agent cortex-github-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Keys a tool's per-round doom-loop identity on the set of fingerprints seen in the round instead of the last call, which genuinely closes the repeated-fan-out gap. But because the set is compared after every call while the round is still filling, a round that is a strict superset of the previous round transiently matches it and can fire observe/block/stop on a call that represents real progress — order-dependently, and untested.

Findings (5)

🟠 major · packages/agent/src/lib/doom-loop.ts:994-998
Mid-round set comparison makes a superset round match the previous round transiently. With r0=[a,b], r1=[a,b], r2=[a,b,c], the b call of r2 sees set [a,b] == prior [a,b] and scores streak 3 -> block, so a legitimate call is refused via checkDoomLoopBeforeExecution/runToolWithHooks (model-result.ts) even though the round added new work. On main nothing fires. The behavior is also emission-order dependent (r2 emitted as [c,a,b] fires nothing), contradicting the PR's 'ordering within the round does not matter'. Expanding fan-outs ([a], [a,b], [a,b,c], …) likewise accumulate streaks they should not.

🟡 minor · packages/agent/tests/unit/doom-loop-fanout.test.ts:150-166
The suite tests the strict-subset direction ('a strict subset is not a repeat') but not the superset direction, which is the case that actually produces a verdict. Add [a,b], [a,b], [a,b,c] asserting no verdict on the middle call, plus an order-permuted variant of the same superset round.

🟡 minor · packages/agent/src/lib/doom-loop.ts:907-914
restore() sets priorStreak: entry.streak, but round is intentionally left undefined, so isSameRound can never be true on the first resumed record and that field is never read — dead state that suggests a semantic it does not have.

🟡 minor · packages/agent/src/lib/doom-loop.ts:1035-1041
Docs are now inconsistent with the implemented semantics: the recordToolCall docstring still states 'A different fingerprint for the same tool resets the streak to 1', the file-header 'Round-scoped streaks' bullet and DoomLoopVerdict.streak ('consecutive identical-fingerprint round count') describe last-call keying, and the verdict message tells the model the tool was invoked 'with identical arguments (fingerprint X…)' — false for a distinct-argument fan-out, where the quoted fingerprint is only the completing call. Since the module advertises itself as the cross-port spec, the round-set identity should also be stated for Python/Go ports.

1 more finding(s)

🟡 minor · packages/agent/src/lib/doom-loop.ts:992-998
At the block rung only the call that completes the set is refused; the other N-1 calls of a repeating fan-out still execute every round (they report the pre-match streak). Detection now works, but mitigation is partial until the stop rung is reached — worth documenting so users do not expect a blocked fan-out to stop spending.

devin-ai-integration[bot]

This comment was marked as resolved.

@perry-the-pr-reviewer perry-the-pr-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perry's Review

Verdict: 💬 Comments / questions
Risk: 🟢 Low

Details

Summary

This PR fixes a real gap in doom-loop detection: streaks were keyed on a tool's last fingerprint, so a repeating fan-out of distinct arguments (read(a), read(b), read(c) reissued verbatim every round) never accumulated evidence. The fix rekeys a round's identity on the set of fingerprints it was called with, compared across rounds. I traced the core logic through every test scenario (distinct fan-out, order-insensitivity, membership change, strict subset, single-call rounds, in-round duplicates, resumed streak) and the streak math is correct in each case. The new fields (roundFingerprints, priorRoundFingerprints, priorStreak) are run-local and never enter getState(), so persisted state is byte-identical — confirmed by reading the serialization path. duplicateInRound semantics also improved: it now catches non-consecutive duplicates within a round (e.g. read(a), read(b), read(a)), which the old last-fingerprint comparison missed.

One suggestion

The restore() path seeds roundFingerprints with only the last call's fingerprint ([entry.fingerprint]) and leaves priorRoundFingerprints absent. For a single-call streak this is fine — the resumed first call matches and increments. For a fan-out streak, the full round set is lost on serialize/resume, so the streak resets to 1 instead of continuing from the persisted value. This is consistent with the PR's stated design ("run-local, never serialized"), but it means a doom loop that spans a serialize/resume boundary (e.g. an approveToolCalls pause) gets a fresh 3-round grace window before re-detecting. A test documenting this behavior — that a resumed fan-out streak resets while a single-call streak continues — would pin the intended semantics for future readers and catch any accidental change.

Notes

  • CI is all green (lint, typecheck, unit-tests, e2e-tests, structural-gate).
  • The verdict message for a fan-out shows the latest call's fingerprint (not the full round set), which is a minor display nit but not worth blocking on.
  • The duplicateInRound behavior change (non-consecutive duplicates now count) is an improvement and is handled correctly by the engine's doomLoopRoundDecisions map.

Comment thread packages/agent/src/lib/doom-loop.ts Outdated
fingerprint: entry.fingerprint,
streak: entry.streak,
// round intentionally absent: first resumed record increments.
roundFingerprints: [

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

▶ Prompt for agents

Suggestion: add a test documenting the resumed fan-out behavior.

restore() seeds roundFingerprints with only the last call's fingerprint ([entry.fingerprint]) and leaves priorRoundFingerprints absent. For a single-call streak this works (the resumed first call matches and increments — tested). For a fan-out streak the full round set is lost on serialize/resume, so the streak resets to 1 instead of continuing from the persisted value.

This is consistent with the PR's stated design ("run-local, never serialized"), but it means a doom loop that spans a serialize/resume boundary gets a fresh grace window before re-detecting. A test asserting that a resumed fan-out resets to streak 1 (while a single-call streak continues) would pin the intended semantics and prevent future regressions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the test, and the same block has a second issue worth folding in.

restore() sets priorStreak: entry.streak (line 913), but priorStreak is only read at line 997, inside the isSameRound branch — and isSameRound requires previous.round !== undefined, which restore() deliberately leaves absent. So that field is dead on every resumed record. Cortex flagged it independently; I traced the read and it is genuinely unreachable.

One correction on the summary: the review says the streak math is correct including order-insensitivity. That holds for the scenarios the suite covers, but the suite only tests the strict-subset direction. The superset direction is untested and does fire — reproduced against head:

[a,b], [a,b], [a,b,c]  => [["none","none"],["none","observe"],["none","block","none"]]
[a,b], [a,b], [c,a,b]  => [["none","none"],["none","observe"],["none","none","none"]]

Same round, same history, different emission order, different outcome — so order-insensitivity does not hold in general, and read(b) gets blocked in a round that added new work. Details on the two threads at lines 996/998. Flagging since the verdict was "would be APPROVE" partly on that basis.

Both tests (resumed fan-out, and the superset/permuted regressions) will land with the fix. Leaving this open until then.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both done in eb3b51d.

Your test — added restarts a resumed FAN-OUT streak at 1, unlike a single-call streak, asserting exactly the asymmetry you described: a two-call fan-out at streak 2 restores to streak 1, while the existing single-call test still shows 3. Comment explains why the round set stays run-local (persisting it would change the state shape, which this PR guarantees is unchanged) and that a loop spanning a resume gets a fresh grace window.

The dead field — removed. I also split the in-round duplicate tracking into its own seenThisRound field, since the old roundFingerprints was doing double duty as both "the round's identity" and "what we have seen so far", which is what made the dead state easy to miss.

The superset issue from my earlier comment is also fixed in that commit: the engine declares a round's complete set before scoring any of its calls, so order-insensitivity now holds for supersets too, not just the permutations of a fixed set that the suite already covered. Worth a look if you revisit — the verdict summary's claim about the streak math was based on the tests that existed, and there are now four more.

CI green, 758 tests (5 new).

The fan-out fix compared a round's fingerprint set while that set was still
filling, so a round that is a strict superset of the previous one transiently
equaled it. With r0=[a,b], r1=[a,b], r2=[a,b,c], the `b` of r2 saw [a,b], matched
the prior round, and scored streak 3 -> block: a call in a round that had added
new work was refused. It was also emission-order dependent — r2 as [c,a,b] never
formed the matching prefix and fired nothing — which contradicted the
order-insensitivity the previous commit claimed. Expanding fan-outs ([a], [a,b],
[a,b,c], …) accumulated streaks the same way.

The engine now declares a round's complete set before any of its calls is scored
(`declareRound`, called from all three execution-batch boundaries), so the
comparison is always whole-round against whole-round. Ordering within a round no
longer matters in fact rather than only in intent, and neither a subset nor a
superset is a repeat — a round that adds work is progress.

Every call in a repeating round now reports that round's streak rather than only
the call completing the match. At the block rung a repeating fan-out therefore
stops spending, instead of executing N-1 of its calls every round.

An undeclared round falls back to per-call sets: exact for single-call rounds,
and for a fan-out no stronger than the pre-fix last-call behavior — a test pins
that it can only reach the hook-only `observe` rung, never refuse a call.

Also: drop `priorStreak` from restore(), which was unreachable (it is only read
under `isSameRound`, and restore() intentionally leaves `round` undefined); add
the resumed-fan-out test Perry asked for; and correct the docstrings, file-header
port spec, and verdict message that still described last-call keying.

Unchanged: single-call round timing, in-round duplicate collapsing, resumed
single-call streaks, persisted state shape, and verdict payloads.

Verified: 758 pass (5 new), typecheck and biome clean. The three new
superset/order/expanding tests fail against the previous commit with the buggy
values ('block' on the progressing call, and order-dependent outcomes).
devin-ai-integration[bot]

This comment was marked as resolved.

Two regressions from the previous commit, both found by Devin's re-review.

A brand-new call could be reported as a repeat. Round-scoped scoring had every
call in a round reuse the round's streak, which is only sound when the round's
membership was declared up front. Server-tool records go through
checkDoomLoopForResponse undeclared, so with one web_search in round R-1 (query
x) and two in round R (x, then a new y), y inherited x's streak of 2 and emitted
a verdict quoting y's own fingerprint and claiming y had been issued in 2
consecutive rounds. Undeclared rounds are now scored per call against the
previous round — the pre-fan-out semantics — so a fan-out there goes undetected
rather than mis-scored. That required restoring priorRoundFingerprints/priorStreak
(dropped last commit as dead) to hold the previous round's baseline for the
length of the current one; they are live on this path and still never serialized.

The steer rung could inject N duplicate corrections for one round. queueDoomLoopSteer
dedupes by exact message text, and every call of a repeating round now emits a
verdict, so interpolating the individual call's fingerprint made three strings
out of one round of evidence and queued all three. A multi-call round now quotes
the round's identity (identical for all its calls) and names the call count;
single-call messages are unchanged.

Also corrects the fallback comment, which claimed the undeclared path degrades
"never to a false positive" — the inheritance bug above was exactly that.

Not changed: Devin also notes that a call repeating inside a round whose other
members vary ([a,b], [a,c], [a,d]) no longer accumulates, since round identity
requires the whole set to match. Confirmed, but it is the previous commit's
deliberate "a changed member is progress" trade-off rather than a regression
introduced here, and restoring per-call streaks alongside round streaks is a
design change. Raised on the thread for the author instead.

Verified: 759 pass (2 new regressions, both fail against eb3b51d), typecheck and
biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…Key once

Three findings from Devin's third pass.

A call outside the declared set could inherit the round's streak. The shared-
streak branch keyed on the tool having a declaration, not on this call being a
member of it. `declareRound` drops a call whose key material is unhashable
(bigint/NaN/circular), but at record time that call still resolves an identity
through the engine's fallback chain, so it took the round's accumulated count
and could be blocked on its first ever appearance — the same failure mode the
declared/undeclared split exists to prevent, reached by a narrower path. Sharing
is now gated on set membership.

A tool's loopKey ran twice per call. Declaring a round resolves each call's key
material, and the per-call checkpoint resolved it again. `loopKey` is user code:
one that counts or logs saw double the activity, and one returning a fresh value
each time made the declared and recorded identities disagree, hiding that call
from detection for the round. The declaration's resolution is now cached per
call id and reused. The fallback warning still logs per call.

README documented the old last-call semantics. It now describes round-set
identity, reset-on-membership-change in both directions, that a repeating
fan-out gets a verdict per member with one shared steer message, and that
DoomLoopDetected fires once per distinct member rather than once per round.
Added two limits to the "does NOT catch" list: a repeat inside a varying round,
and fan-outs on paths that cannot declare a round (server tools).

Verified: 761 pass (2 new regressions, both fail against 81c2572 with the buggy
values — streak 3 instead of 1, and loopKey invoked 4 times instead of 2);
typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…nd set

Completes the previous commit's fix, which gated READING the declared set on
membership but not WRITING it. A call the declaration could not include —
`declareRound` drops unhashable key material, and the engine still records it via
its fallback chain — stored `roundFingerprints: [itsOwnFingerprint]`, replacing
the round's declared set. The next round's declared member then compared against
that singleton, failed to match, and reset to 1; with the unhashable call
recurring every round the member's streak was pinned at 1 permanently. Measured:
[1,1,1,1] across four repeating rounds where the control climbs [1,2,3,4]. So a
single bigint in one call's arguments disabled doom-loop detection for that tool
for the rest of the run — the inverse of the fail-open guarantee, which allows an
unhashable value to cost detection for its OWN call only.

Devin also noted an order dependence: a non-member recorded before the round's
members made them inherit its streak. Both symptoms had one cause — round-level
state (the round's identity and score) and per-call state (fingerprint, in-round
dedupe) shared one mutable record. They are now written separately: the streak is
computed as a pure function of (this round's set, the previous round's set, that
round's streak), so arrival order cannot affect it, and a non-member records its
own identity while leaving the round's identity and score to its declared
members. The round TRANSITION is still recorded by whichever call arrives first,
so the baseline advances even when a non-member opens the round — fixing that
was what the first attempt at this commit got wrong.

Regression test asserts the member streak climbs 1..4 with an unhashable call
riding along, in BOTH emission orders. Fails against f44c69d with [1,1,1,1].

Verified: 762 pass, typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…d changeset example

Two findings from Devin's fifth pass.

`beginDoomLoopRound` declared every call in the batch, resolving each one's
`loopKey` up front — including calls that are never checked. A manual tool (no
`execute`, no `onToolCalled`) is handed to the caller and never recorded as
evidence, and every execution path skips it via `isAutoResolvableTool`, but the
declaration ran its `loopKey` anyway. For user code that counts or logs inside
that callback, this was activity for a call the detector never evaluated. The
declaration now applies the same `isAutoResolvableTool` predicate, which is also
correct on its own terms: a call that is not evidence is not part of the round.

Note the reproduction needs a MIXED batch. An all-manual round never reaches
`beginDoomLoopRound` at all (`hasExecutableToolCalls` guards it), so the first
version of this test passed with and without the fix — it proved nothing. The
committed test pairs a manual call with an executable one and asserts the
executable call's loopKey runs exactly once while the manual call's never runs;
it fails without the guard with "called 1 times".

The changeset had no code example, which .agents/skills/public-api-examples
requires for behavioral changes to a public option even when the signature is
unchanged. Added one to the changeset and an `### API example` section to the PR
description, both showing the same before/after: a repeating three-call fan-out
that previously never tripped now observes at round 2 and blocks every call of
the round at 3, while a round that adds work resets to 1.

Verified: 661 unit tests pass, typecheck and biome clean. (One e2e cancellation
test failed once on a full run and passed in isolation and on re-run — a live
network timing flake, unrelated to this change.)
devin-ai-integration[bot]

This comment was marked as resolved.

…resume

The persisted state holds one fingerprint and one count per tool, so it cannot
express "this count was earned by the set {a,b,c}". Restoring a fan-out's streak
verbatim attached the whole count to whichever member happened to be recorded
last, so a resumed round consisting of just that one call matched, inherited the
fan-out's evidence, and was BLOCKED on its first appearance — while the model had
done strictly less work than before the save. It was also arbitrary: resuming
with a different member of the same fan-out scored 1 and passed.

`getState` now persists a multi-call round's streak as 1. Under-counting on
resume is the safe direction — the round is re-observed and re-accumulates from a
correct baseline, which the test asserts so the fix cannot silently become a
detection hole.

Measured, 3-call fan-out repeated twice then resumed with one call:
  before this PR (main): saved 1, resumed -> streak 2, observe
  eb3b51d..43b88c2:      saved 2, resumed -> streak 3, BLOCK
  now:                   saved 1, resumed -> streak 2, observe

So the mechanism predates this PR, but making fan-outs accumulate raised the
saved count, which escalated the resumed outcome from a harmless observe to a
refused call. That makes it this PR's regression to fix.

Also corrects the comment in restore(), which claimed a resumed fan-out streak
"restarts at 1" — it did not, and the claim is only true now that getState
enforces it. The existing resume test passed either way because it resumed the
same multi-call set, which never matched; it never covered the single-call case.

Verified: 662 unit tests pass; the new test fails without the getState change.
Typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…ositive class

The changeset asserted two things that were not true.

"Resumed streaks unchanged" — the previous commit deliberately made a multi-call
round's saved streak restart after a resume. The changeset now states that, why
(the persisted shape cannot express which set earned a count), and that
single-call streaks still continue.

"No API surface changed" — `DoomLoopMonitor` is exported from src/index.ts, so
`declareRound` is a new public method. Documented as additive, with a note that
`callModel` users need not touch it while direct `DoomLoopMonitor` users and SDK
ports should, since an undeclared multi-call round's fan-out goes undetected.
Bump raised patch -> minor accordingly: .agents/skills/changeset-versioning
specifies minor for new exports and features.

Also documents a false-positive class this PR newly makes reachable, which is
worth a decision before shipping (raised on the thread, not resolved here).
Because a round's identity is the whole set, a tool called with a stable set of
parallel arguments every round now accumulates where it previously could not.
Measured, an agent re-reading three context files every turn:

  round 1: none  none  none
  round 2: observe observe observe
  round 3: block block block      <- all three reads refused, every round after

That is a legitimate shape, and it produces N synthesized error outputs per
round rather than one. `loopKey: false` is the opt-out; no prior exemption
covered this, since the shape was invisible to the detector before. Added to the
README next to the `loopKey` exemption guidance and to the changeset.
devin-ai-integration[bot]

This comment was marked as resolved.

`beginDoomLoopRound` declared some calls that never reach the doom-loop
checkpoint, so they became phantom members of the round's identity and the
sibling that WAS recorded got scored against a set including them. The streak
then reset the moment the phantom stopped being emitted, even though the recorded
call never changed.

Two sources, both closed:

- The malformed-arguments branch ran BEFORE the tool-resolvability check, so a
  raw-string call to an unknown or manual tool was declared despite never being
  recorded. The tool lookup and `isAutoResolvableTool` gate now precede it.
- A call the PermissionRequest hook denied without pausing: `hookDeniedCalls` is
  populated before the round begins and `runToolWithHooks` synthesizes the
  rejection before the checkpoint, so those are skipped too.

The new test drives the monitor directly with an over-broad declaration to pin
the consequence — an identical recorded call scores [1,2,1,2] across four rounds
when a phantom member is present for the first two — so the reason for the
engine-side filtering is documented rather than implicit.

Note on the test: an earlier version of this drove the engine end-to-end with a
malformed manual call, and produced ZERO detections — the loop pauses on the
manual call before later rounds run, so it asserted nothing. Removed rather than
patched; the monitor-level test verifies the actual mechanism. That is the third
test this session that would have passed against the bug it claimed to cover, so
I am now deriving the expected numbers before writing the assertion instead of
after.

Verified: 663 unit tests pass, typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…t its earner

Two findings from Devin's ninth pass.

`resolveLoopKeyMaterial` can throw, and both call sites were unguarded. It
catches a throwing `loopKey`, but the field-list form does `field in args` and
`args[field]`, so a getter or proxy trap on the arguments object escapes it.
Declaration resolves the whole batch up front, so an uncaught throw there would
fail the round and the run over one odd call — the opposite of the invariant that
detection only ever affects a run through a ladder action. Both sites now skip
just that call and warn.

Severity is narrower than reported, and worth recording: this is NOT reachable
through `callModel`. Tool arguments come from `JSON.parse` (stream-transformers),
so they are always plain objects, and `PreToolUse` argument mutation happens
after the round is declared. It is reachable for direct callers, since both
`resolveLoopKeyMaterial` and `DoomLoopMonitor` are exported, and for ports that
build key material differently. Guarded regardless.

`getState` paired the saved streak with the wrong identity. `fingerprint` is what
pairs with `streak` in persisted state, and a non-member recorded LAST in a round
overwrote it, so the count was attached to a call that never earned it. Both
halves broke on resume: the non-member (a call detection is meant to ignore)
matched, inherited the count, and was BLOCKED on its first appearance, while the
genuinely repeating call reset to 1 and lost its evidence. Measured, saved streak
2: ignored call -> streak 3 block / real repeat -> streak 1; now -> streak 1 none
/ streak 3 block. A non-member no longer overwrites the identity; it is still
tracked for in-round dedupe via `seenThisRound`.

Test-quality note: my first attempt at the resolution-throw test passed WITHOUT
the fix, because the end-to-end case I chose (loopKey returning a bigint) is
caught by the pre-existing fingerprint fallback and never reaches the new throw
path. Rewritten to pin the throw directly and to state in-comment that the engine
path cannot reach it. That is the fourth test in this PR that would have passed
against its own bug; every assertion here was derived from a measured run and
verified to fail with the fix removed.

Verified: 665 unit tests pass, typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…areRound example

Devin traced that undeclared multi-call rounds do NOT behave "the same as
before", and the trace is right. Verified:

  undeclared [a,b] x6:  b -> 1, 2 observe, 3 block, 4, 5, 6 stop
  order flipped:        the verdict moves to the OTHER call
  [a,b],[c,b],[d,b]:    b -> 3 block

Each call of an undeclared round overwrites `roundFingerprints` with its own
singleton, so the next round's matching call compares against the previous
round's LAST recorded fingerprint. A repeating undeclared fan-out therefore does
accumulate — on whichever member lands last, order-dependently — and it reaches
`stop`. The changeset, the README limit, and a source comment all claimed such
fan-outs go undetected. Corrected all three, and added a test pinning the real
behavior (including that a repeat inside a VARYING round accumulates here, which
the declared path treats as progress).

No behavior change: the engine declares every executed batch, so this is the
server-tool and direct-caller path only.

Also addresses the changeset's missing example for the new public method. While
writing it I ran it, and it did not work: `resolveDoomLoopOption` is not
exported, so the obvious construction fails at runtime. `DoomLoopMonitor` is
exported but a consumer must hand-build the resolved config shape to instantiate
it. Rewrote the example to only use exported API and noted the export gap as a
follow-up — it predates this PR and is unrelated to `declareRound`. Also dropped
the "No API surface changed" line, which was still there from before the bump was
raised to minor.

Verified: 666 unit tests pass, typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…ructible

`DoomLoopMonitor` was exported without its config resolver, so the natural
construction — `new DoomLoopMonitor(resolveDoomLoopOption(true))` — failed at
runtime for any consumer: `resolveDoomLoopOption` and `ResolvedDoomLoopConfig`
existed only at module level. The class was effectively unusable outside
`callModel` short of hand-building the internal resolved-config shape. Found by
executing the changeset's usage example instead of eyeballing it.

Exports `resolveDoomLoopOption` (value) and `ResolvedDoomLoopConfig` (type) from
the package entrypoint, and adds a consumer-contract test file that imports from
`src/index.js` only — construction with defaults, a custom ladder, fan-out
detection via declareRound, and a JSON state round-trip across a simulated
process boundary. Changeset example updated to match and the follow-up note
removed, since this was that follow-up.

Already covered by the existing minor bump.

Verified: 669 unit tests pass (3 new), typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

… identities

Behavior-identical simplification of the scoring path — all 669 tests pass
unchanged, including the 20 fan-out regressions that were each verified to fail
against the bug they cover.

The branching had accreted one special case per bug fix: a member/non-member
fork for the reported streak, a second fork recomputing the stored round streak,
a third choosing the persisted identity, each with its own baseline reads. All
of it reduces to a single scoring rule applied to two identities:

  score(set) = baseline matches set ? priorStreak + 1 : 1

  reported streak   = score(callSet)    callSet  = declared set if member,
                                                   else the call's singleton
  stored round state = score(roundSet)  roundSet = declared set if one exists,
                                                   else the call's singleton

The baseline (previous round's set + streak) is fixed at the round transition
and read once. Every non-member rule from the last several commits falls out of
the callSet/roundSet distinction instead of being its own branch: a non-member
scores 1 because its singleton is not the baseline; it cannot clobber the round
because roundSet prefers the declaration; the round transition still advances
because the baseline write is unconditional; the persisted identity guard is the
one remaining explicit special case.

Net -70 lines in the hot path. The verdict message now derives from callSet,
which is what the call was actually scored with (same value as before in every
reachable case).

Also spot-checked beyond the suite: non-member-first ordering across four
rounds, in-round duplicate handling, and the resume identity pairing all produce
byte-identical traces to the pre-refactor code.
devin-ai-integration[bot]

This comment was marked as resolved.

…e/resume

The persisted shape carried one (fingerprint, streak) per tool, which cannot say
WHICH set earned a count. That forced a choice between two failure modes, and
this PR had cycled through both: persist the streak verbatim and a resumed
subset call inherits a fan-out's whole evidence (blocked on first appearance —
the I1 false positive); persist 1 and the evidence is discarded at every save.
Devin's last two passes showed the second mode is worse than the changeset
admitted: saveStateSafely snapshots on every persist, so an approval/HITL pause
reset a fan-out sitting at the block rung, and per-turn-resume topologies (one
callModel per user turn — the serverless pattern) never accumulated at all.

A multi-call round now persists its full fingerprint set (optional
`roundFingerprints` on `DoomLoopStreak`, additive). The streak travels with the
exact set that earned it, so both failure modes are gone rather than traded:

  per-turn resume, identical 3-call fan-out:  1 -> 2:observe -> 3:block -> 4  (was 1,1,1,1)
  pause at block rung, resume, repeat:        4:block                         (was reset)
  resume with a SUBSET of the saved set:      1, no verdict                   (unchanged)

Compatibility: single-call rounds omit the field (their fingerprint fully
describes the round), pre-existing blobs restore with their old semantics, and a
malformed persisted set (non-string entries) degrades to the lone fingerprint
instead of dropping the entry. Text streaks never carry it.

Also overloads resolveDoomLoopOption so `new DoomLoopMonitor(
resolveDoomLoopOption(true))` — the changeset's own example — compiles under
strict TS: a `true`/config argument now types as non-null, while the engine's
pass-through of a raw caller option keeps the nullable signature. Devin flagged
the example as non-compiling; verified with a strict-mode tsc run before and
after.

Tests: the two resume tests now pin continuation instead of the old downgrade
(both fail against the previous commit), a public-API test drives the serverless
per-turn pattern end to end through JSON, and a legacy/hostile-blob test pins
backward compatibility. 671 unit tests pass, typecheck and biome clean.
…laration site

Devin verified all three record-skip paths are mirrored by the declaration's
filters (isAutoResolvableTool, hookDeniedCalls, loopKey exemption) and flagged
the coupling as fragile: a future short-circuit added between declaration and
the checkpoint would silently degrade fan-out detection for that tool. The
invariant, its failure mode, and the mirror list are now stated where the next
edit will happen, pointing at the regression test that pins the consequence.
devin-ai-integration[bot]

This comment was marked as resolved.

…asing it

The persisted `roundFingerprints` was the live array — shared with the running
streak entry and, for declared rounds, with the round declaration itself — while
every other field in the snapshot is a primitive copy and the caller's
StateAccessor receives the blob directly. Measured: pushing one element into the
saved snapshot silently corrupted the running detector, and the next identical
fan-out scored 1:none instead of 3:block for the rest of the run.

Copy on write-out. restore() already copied (and sorted) its input, and
declareRound's sets are built internally, so getState was the only aliased
surface. Regression test mutates a saved snapshot and asserts the live detector
still blocks; fails against the previous commit with 1:none.

Verified: 672 unit tests pass, typecheck and biome clean.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Open in Devin Review

Comment thread packages/agent/src/index.ts
Comment on lines +1139 to +1149
const priorSet = isSameRound
? previous?.priorRoundFingerprints
: (previous?.roundFingerprints ??
(previous
? [
previous.fingerprint,
]
: undefined));
const priorStreak = isSameRound ? (previous?.priorStreak ?? 0) : (previous?.streak ?? 0);
const score = (set: readonly string[]): number =>
priorSet !== undefined && setsMatch(priorSet, set) ? priorStreak + 1 : 1;

@devin-ai-integration devin-ai-integration Bot Jul 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Undeclared multi-call rounds now match anywhere in the round, not just in the final position

The fixed-at-transition baseline changes undeclared (server-tool / direct-caller) semantics relative to the old last-fingerprint comparison: previously the second call of a round was compared against the first call of the same round, so [a] then [b, a] scored a as 1; now a is compared against the round-transition baseline [a] and scores 2 (observe). This is documented in the README limits section and is arguably more correct (a genuinely recurred in consecutive rounds), but it does mean server-tool verdicts can now fire one round earlier for some emission orders, and server-tool streaks can reach stop.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed against pre-PR main and documented in f8584f5 — your trace is exact:

round0=[a], round1=[b,a], undeclared:
  main (78c562e): b=1, a=1   (b overwrote the last fingerprint; a reset)
  this PR:        b=1, a=2   (baseline fixed at the round transition)

Intentional, and I am keeping it, for three reasons:

  1. It is strictly more detection with no new false-positive path. Every increment still requires the fingerprint to have been genuinely issued in the previous round — the fixed baseline only stops a mid-round sibling from DESTROYING evidence by overwriting the comparison point. a really did repeat across consecutive rounds; the old 1 was the accident, not the new 2.
  2. The fixed baseline is what makes declared rounds order-independent. Reverting the undeclared path to "compare against whatever was recorded immediately before" would mean two baseline mechanisms, and the last several bugs on this PR all lived in exactly that kind of dual-path divergence.
  3. Order-dependence remains, but only in the harmless direction: which member accumulates still depends on what the PREVIOUS round recorded last, but within the current round a match now counts anywhere rather than only in final position — less order-sensitive than before, not more.

The README bullet now spells out the mid-round match ([a] then [b, a] scores a as a repeat even though b arrived first) alongside the last-member shape it already described. Declared rounds — every batch the tool loop executes — are unaffected either way.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and documented in the README limits section. Leaving open deliberately: this is a semantic change to undeclared (server-tool / direct-caller) rounds rather than a defect, and it pairs with the scope question on the per-call detector below. Flagging both for a human call before merge rather than resolving unilaterally.

…lared path

Devin traced that undeclared rounds no longer reproduce the old per-call
comparison exactly, and the trace is right. Verified against pre-PR main:
round0=[a], round1=[b,a] undeclared — old code reset a to 1 (b had just
overwritten the last fingerprint); the baseline is now fixed at the round
transition, so a scores 2. Strictly more detection (every increment still
requires the fingerprint to have been genuinely issued in the previous round),
no new false-positive path, and declared rounds are unaffected — but the README
bullet described only the last-member shape, so the mid-round match is now
spelled out there too.
…s change

Round-set identity treats any membership change as progress, which is right for
the fan-out as a unit but blind to one call repeating inside varying company:
[a,b], [a,c], [a,d] never accumulated on `a`, a paused HITL member granted the
resumed round a fresh grace window, and undeclared multi-call rounds (server
tools) accumulated only on the last-recorded member, order-dependently.

Each (tool, fingerprint) identity now counts its own consecutive rounds
alongside the round-set streak, and the stronger evidence decides:

  [a,b],[a,c],[a,d]           a -> 1, 2:observe, 3:block; b/c/d run free
  HITL member drops on resume  the repeat keeps counting: 1, 2:observe, 3:block
  undeclared [a,b] x2          both members 2:observe (was: last-recorded only)
  undeclared [b,a] flipped     identical outcomes (was: verdict moved calls)
  [a,b,c] x2 then [a,b,c,d]    a,b,c 3:block (each genuinely re-read), d executes
  exactly-repeating round      both counts equal — nothing double-fires

The verdict message follows the evidence: a per-call-only verdict quotes that
call's own identity (dedupe-safe — no other call carries the same text), while
round verdicts keep the shared round-identity message for the steer rung.

Persistence gains `callStreaks` (optional, additive) next to `roundFingerprints`
so per-call evidence survives approval pauses and per-turn resumes; old blobs
restore with their old semantics, and a fresh-after-resume call still inherits
nothing (the count follows who EARNED it — any member of a saved fan-out resumed
alone continues its own 2 -> 3, a never-recorded call starts at 1).

Semantics deliberately changed from the previous commits, with the failing
expectations updated rather than preserved: a membership change now flags the
calls that DID repeat instead of resetting everything (subset rounds observe
their re-issued calls; superset rounds block the repeated members while the new
call always executes — order-independently, unlike the original superset bug,
which blocked the NEW work). Three known limits this closes were documented in
the README/changeset as recently as yesterday; those entries are replaced by the
new semantics, and the remaining honest limit (cross-tool alternation, measured:
an every-other-round repeat still accumulates, slowly) is documented instead.

11 of the 21 fan-out tests fail without the feature (verified by stashing the
implementation); the engine-level suites pass unchanged. 673 unit tests,
typecheck and biome clean.
devin-ai-integration[bot]

This comment was marked as resolved.

…tructural gate

The per-call-streak commit tripped the structural gate ("complex functions 9 ->
10"): the three-way message ternary pushed recordToolCall past the complexity
threshold. The message construction is self-contained — five inputs, no state —
so it moves to a named builder, buildToolVerdictMessage, which also gives the
steer-dedupe rationale a proper docstring instead of an inline comment.

recordToolCall's branch count returns to its pre-feature level; behavior is
byte-identical (673 unit tests pass unchanged, message strings not touched).

The e2e failure on the same run is unrelated: multi-turn-tool-state's
"preserve original user input" timed out against the live API and passes
locally in 30s; re-run requested.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment on lines +1287 to +1294
/*
* The stronger of the two detectors decides. The round streak covers the
* reissued fan-out as a unit; the per-call streak covers a repeat whose
* round-mates keep changing. For an exactly-repeating round both counts
* are equal, so nothing double-fires — the counts only diverge when one
* detector sees something the other cannot.
*/
const effectiveStreak = Math.max(streak, callStreak);

@devin-ai-integration devin-ai-integration Bot Jul 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Per-call streaks are a second, independent detector that fires without any declaration — new detections beyond the fan-out fix

The PR title/description frames the change as "detect a repeated same-tool fan-out", but the shipped implementation adds a second detector: callStreaks at packages/agent/src/lib/doom-loop.ts:1227-1233 counts every (tool, fingerprint) identity's consecutive rounds independently, and Math.max(streak, callStreak) at packages/agent/src/lib/doom-loop.ts:1294 lets it decide alone. This changes verdicts on paths the fan-out work does not touch: an undeclared multi-call round (server-tool records via checkDoomLoopForResponse in packages/agent/src/lib/model-result.ts:1844, and any direct DoomLoopMonitor consumer) previously accumulated only on whichever call was recorded last, and now every repeated member accumulates. It also means a round that adds new work ([a,b][a,b,c]) still blocks a and b from round 3, so an agent that re-reads a stable anchor file while making genuine progress is refused unless the tool opts out with loopKey: false. The changeset does document this as a newly reachable false-positive class, but it is worth a deliberate product decision rather than being read as a side effect of the fan-out fix.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed on every particular, and worth stating plainly: this is NOT a side effect — it is a deliberate product decision, made explicitly by the author, and the framing you are asking for exists on the record.

The sequence: I documented the varying-round and HITL gaps as known limits and put the per-call-streak design (originally your suggestion, on the varying-round thread) to the author as a build/no-build decision, including the false-positive cost. The author's direction was to build it, with the reasoning that a truly identical repeated call SHOULD flag — re-reading a file that is already in context is waste, not progress, and a deliberately-repetitive tool declares itself with loopKey: false/null. The stable-anchor-read shape you name was specifically weighed and accepted on those grounds.

Your undeclared-path observation is also right and also intended: per-call evidence needs no declaration, which is precisely what fixed that path's order-dependence (previously only the last-recorded member accumulated; now every repeated member does, emission-order-independently).

On the title framing: fair. The PR grew from "detect a repeated fan-out" to "two-detector design" across the review; the PR description's implementation note and the changeset both describe the per-call detector, its verdict-message behavior, and the false-positive class with the opt-out. I have kept the changeset as the authoritative consumer-facing description since it is what ships to the changelog.

No code change from this thread; the decision trail is the answer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right question and I am escalating it rather than resolving it. Confirmed at head: callStreaks counts every (tool, fingerprint) identity independently and Math.max(streak, callStreak) lets it decide alone, so it does change verdicts on paths the fan-out work does not touch — including the [a,b][a,b,c] case where a stable anchor file re-read gets refused while genuine progress is being made.

Two options for the human: (1) ship as-is and retitle, since the PR currently claims only the fan-out fix while shipping a second detector; or (2) gate the per-call detector behind an opt-in and land the fan-out fix alone, deferring the broader detection. Given this feature is slated to be announced on its own day as a first step, option 2 is the more conservative launch. Leaving open for that decision.

… function

The previous gate fix extracted the wrong function. I estimated complexity with
a crude branch-token count over recordToolCall and shipped without running the
gate; the actual offender was restore(), which the per-call restore validation
had pushed to cc=18. Found by running sentrux locally this time (a darwin binary
exists in the same release) — `sentrux check` names the functions, which the
gate summary does not.

The per-entry restoration moves to restoreStreakEntry: set validation, per-call
count validation, and the legacy-blob fallback, with the restore-semantics
rationale as its docstring. restore() drops to a guard plus a loop. Verified
against the real gate: "No degradation detected", complex functions back at the
baseline 9 (all pre-existing, none in doom-loop.ts).

Behavior byte-identical: 673 unit tests pass unchanged.
devin-ai-integration[bot]

This comment was marked as resolved.

…ext; drop dead id guards

Three findings from Devin's pass over the per-call feature.

Per-call evidence was dropped at exactly the moment it was the only evidence.
getState omitted `callStreaks` for single-call rounds on the assumption that
`fingerprint`+`streak` carry the same information — false when a round SHRINKS:
a paused HITL member drops out, the round streak resets to 1, and the per-call
count (2) is the only record that the surviving call repeated. Saving then
restoring handed the repeat a fresh grace window, reaching block one round later
than the in-memory behavior the HITL test pins. The omission now fires only when
the per-call map is exactly {fingerprint: streak} — the case restore()
reconstructs verbatim. Measured: resumed streak 2 -> 3 (was 2 -> 2).

A wide round could queue one steer message per repeated member. Per-call
verdict text embedded the call's own fingerprint, so a 20-wide fan-out repeated
and then widened produced 20 distinct messages — all queued, all joined into one
injected prompt. Per-call messages no longer embed the hash: same tool + same
count is byte-identical, and the steer dedupe collapses the round to one
correction (measured 20 -> 1). The refused call is still identified by its block
output and the verdict payload's `fingerprint`; only the prose lost the hash.

The loopKey cache's id guards were dead. ParsedToolCall.id is a required
string, so `toolCall.id !== undefined` never gated anything and the "silently
falls back when a call carries no id" path was unreachable — but the guards
implied it existed. Removed them and keyed the map on the id directly, so the
type system now states what the guards obscured.

Both behavior fixes have regression tests verified to fail against the previous
commit. 675 unit tests, typecheck, biome, and the structural gate all clean.
@LukasParke

Copy link
Copy Markdown
Contributor Author

Re the CHANGES_REQUESTED review at f46f74c — all five findings were addressed in the 20 commits since; head is now d956562 (main merged in). Detail:

🟠 major — mid-round superset match. Fixed by eb3b51d ("score a doom-loop round's declared set, not a growing prefix"). Scoring is no longer done against a filling prefix, so [a,b], [a,b], [a,b,c] no longer blocks the b call. Three tests guard it: flags the repeated members of a superset round, never the new one, scores a superset round the same whatever order it is emitted in, and the expanding-fan-out case (doom-loop-fanout.test.ts:206-276).

🟡 missing superset test coverage. Added by the same commit — including the order-permuted variant you asked for.

🟡 restore() priorStreak dead state. priorStreak is live now, read at doom-loop.ts:1224 to carry a prior round's count across the same-round check.

🟡 stale docs / verdict message. The verdict builder branches on callSet.length > 1 and emits "the same set of N parallel calls (round identity …)" for fan-outs; "identical arguments (fingerprint …)" is only used for genuine single-call repeats, so it is no longer false. Round-set identity is documented in the README as the cross-port contract.

🟡 partial mitigation at the block rung. Now inaccurate as written — every declared member of the round receives the verdict, not just the completing call, and the README states this explicitly ("because every call in the round gets the verdict, that is N synthesized error outputs per round, not one").

Verification at head: 86/86 doom-loop tests pass, typecheck and lint clean. Two failures in hooks-contract-fixes.test.ts (vi.advanceTimersByTimeAsync is not a function) reproduce on clean origin/main at d030602 and are unrelated to this branch.

Two threads left open deliberately for a human decision — the per-call detector's scope, and the undeclared-round semantics change. Those are product calls, not defects.

@LukasParke

Copy link
Copy Markdown
Contributor Author

Correction to my previous comment: the two hooks-contract-fixes.test.ts failures I mentioned were an artifact of my running bun test directly instead of the repo's runner (bun run test → turbo → vitest). Under the correct runner this branch is fully green at d956562:

  • @openrouter/agent: 53 files, 675 tests passed, no type errors
  • @openrouter/mcp: 9 files, 39 tests passed, no type errors

No pre-existing failures on main either — that claim was wrong and I'm retracting it. Lint clean, typecheck clean.

devin-ai-integration[bot]

This comment was marked as resolved.

The steer rung dedupes queued guidance on exact message text, so a round
of evidence must render one string. On the undeclared path (server-tool
records, direct DoomLoopMonitor consumers, the SDK ports) each call is
recorded alone, so callSet holds only that call however wide the round
was. The last-recorded call tied at roundStreak == callStreak and took
the fingerprint-bearing single-call branch while its round-mates took the
per-call branch — two near-identical corrections queued for one round.

buildToolVerdictMessage now takes roundDeclared: only a declared round's
callSet describes the round, so only then may the text name a set or
quote an argument fingerprint. Declared fan-outs and declared single-call
rounds are unchanged.

Not reachable via callModel, where every executed batch is declared.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant