feat(agent): doom-loop detection for the tool-execution loop - #73
Conversation
Opt-in via doomLoop on callModel. Deterministic detection of runs that stop making progress: consecutive identical tool calls (per-tool fingerprint streaks over tool-declared loopKey identity, incl. repeated empty and invalid-JSON calls) and repeated text tokens (within-response block repetition + cross-step identical-text streaks). Graduated response ladder observe -> steer -> block -> stop, per-event override via the new DoomLoopDetected hook, streaks persisted in ConversationState.doomLoop across serialize/resume, and SessionEnd.reason 'doom_loop' + ModelResult.getDoomLoopVerdict() on stop.
Fingerprints: RFC 8785 (JCS) canonicalization + SHA-256/UTF-8 via WebCrypto replaces cyrb53/UTF-16 (cross-port contract now real; vectors in tests/vectors/doom-loop-fingerprints.json; bigint/NaN/circular/deep key material rejected with engine fallback to full-args identity). Streaks are round-scoped: N identical parallel calls in one round count once and share the round's decision (evaluations serialized in model-emission order under the async hash). loopKey is now function | field-list | false on the tool definition (declarative forms are data — serializable, MCP-transportable); undefined returns fall back with a warning instead of colliding. markMcp accepts a loopKey override. Stop verdicts seal state (synthesized halt outputs for unresolved calls — no dangling function_call 400s on resume), gate the allow-final-response and approval-resume request paths, persist across decision-only resumes, and clear on fresh conversational turns. Text-only no-tools stops report SessionEnd reason doom_loop. Steer guidance queued before a pause persists and delivers on resume. Server tools fingerprint at the step checkpoint (observe/steer/stop). Ladder configs warn on dead rungs and block-without-stop. Documented, test-locked misses: nonce-varying args without loopKey, paraphrased text.
The committed baseline (coupling 0.43, 133 import edges) predates the mcp package landing — origin/main itself measures 0.47 with 306 edges against it, passing only within tolerance. The doom-loop feature's real structural delta over current main is +0.012 coupling (0.47 -> 0.49, the new lib/doom-loop module's fan-in from model-result, tool-types, tool, async-params, index, and the mcp wrapper), which is within the gate's tolerance. Baseline regenerated from origin/main (61a2a9a) via sentrux v0.5.7 gate --save, matching the CI binary version. Cycle count (1, in the mcp package) and complex-function count (9) are pre-existing and unchanged by this branch.
Fix 3 (SDK): new 'signal' option on callModel — aborting stops the
tool loop at the next turn boundary AND aborts the in-flight request/
stream, rejecting with the abort reason; pre-aborted signals fail
before any dispatch. RequestOptions.timeoutMs now reliably bounds EACH
request even when a signal is present: the SDK skips its timeoutMs
wiring whenever a request carries a signal, so the engine composes
{run signal, caller signal, fresh per-dispatch AbortSignal.timeout}
via AbortSignal.any at every send site (initial, tool rounds, final,
retry, resume). 7 new unit tests (fake-timer driven) pin the composed
semantics.
Fixes 1+2 (DEV-658 e2e deflake): measured healthy path was 70-80s
across three GLM requests — the final turn alone burned 50-65s on a
750-1150-token reasoning burst — leaving no margin under the old 120s
budget (one CI window timed out attempt AND retry back-to-back).
Now: 300s budget, bounded generation (reasoning.maxTokens 512 +
maxOutputTokens 1024), a shallow unanswerable-without-search prompt
that preserves the leak pressure without inviting multi-hop parametric
reasoning, and a 90s per-request timeoutMs so a stalled provider fails
fast and the vitest retry gets a fresh draw. Verified 3/3 live passes
at 10-20s each.
The per-option conditional-spread chain pushed callModel to cc=16 (sentrux max 15) once the signal option landed. Build the options object once and strip undefined keys — identical absent-key semantics, one loop instead of thirteen branches.
There was a problem hiding this comment.
Summary
This update's only functional delta versus the prior reviewed revision is a cosmetic refactor in call-model.ts (building engineOptions via a plain object + a delete-if-undefined loop instead of per-key conditional spreads) — behaviorally equivalent, no new risk. All doom-loop logic (doom-loop.ts, model-result.ts doom-loop wiring) is byte-identical to what I reviewed last round, so the one outstanding major finding from that review is still unresolved. Everything else (round-scoped streaks, cross-port fingerprint contract, seal-on-stop, steer/condemnation persistence, cancellation/timeout composition) remains as previously assessed: well-designed and heavily tested.
Findings (2)
🟠 major · packages/agent/src/lib/doom-loop.ts:600-650
OPEN (unresolved from prior round): DoomLoopMonitor.restore() builds tools: Record<string, StreakEntry> = {} (a plain object with the default Object.prototype) and assigns tools[name] = {...} for every key in the untrusted persisted blob after only checking isValidStreak(entry). A persisted ConversationState.doomLoop containing {"tools":{"__proto__":{"fingerprint":"x","streak":999}}} passes that check and pollutes the object's prototype via the bracket-notation setter, so this.tools[anyUnseenToolName] subsequently inherits the attacker-seeded streak instead of returning undefined — corrupting streak bookkeeping (false-positive stop / detection bypass) for every tool once that state round-trips through a resume. This code region is unchanged since the previous review pass (the remediation commit did not touch it); still needs a name === '__proto__' / 'constructor' / 'prototype' guard or Object.create(null).
⚪ nit · packages/agent/src/inner-loop/call-model.ts:140-175
Refactor from per-key conditional spreads to building engineOptions as a mutable record and deleting undefined-valued keys is functionally equivalent (verified against GetResponseOptions consumers, which all use optional chaining / !== undefined checks, not in/hasOwnProperty). No behavior change; no action needed.
The unit suite pins the signal/timeout composition against mocks; these five e2e tests prove what mocks cannot — the composed abort signal reaches the real fetch/stream. Covered: pre-aborted signal fails with zero dispatches; aborting mid-generation kills a live request fast (bounded wall-clock assertion, not exact timing); per-request timeoutMs bounds a live request even with a run signal present (the SDK-disabling configuration DEV-658 relies on); abort during tool execution stops before the follow-up dispatch; and a composed-signal run completes normally when neither bound fires. Verified 3/3 locally (~8s a run).
Unit: drop fake timers around AbortSignal.timeout — vitest fake timers cannot fake its Node-internal timer, so advancing the mocked clock proved nothing (a WRONG shared per-run timer would never fire in near-zero real time either, passing the per-dispatch test vacuously). Real tiny budgets (50-120ms, documented exception) make the two timeout tests actually discriminate; suite cost ~215ms. E2E: assert rejection IDENTITY, not just 'anything threw' — the mid-flight abort test walks the cause chain to OUR abort reason (the SDK wraps it in UnexpectedClientError), and the timeoutMs test matches the TimeoutError DOMException, so an unrelated transport/auth failure can no longer pass either test. Verified live: rejection shapes probed against the real API; 5/5 passing.
…tore
Review finding (cortex): DoomLoopMonitor's streak store was a plain
object; restoring a persisted blob {"tools":{"__proto__":{...}}} from
client-writable state storage reassigned the store's prototype, so
every unseen tool inherited the seeded streak (false stop verdicts /
detection bypass). The store is now a Map — hostile keys are inert
data, and a tool legitimately named __proto__ works. getState() keeps
emitting a plain-JSON record via Object.fromEntries (own-property
defines, no setter hits). Regression tests for both cases.
Also corrects the WebCrypto doc claim (Devin): globalThis.crypto is
unflagged in Node >=19, not >=18 (18 needed
--experimental-global-webcrypto and is EOL); supported floor is the
active LTS (CI runs Node 22).
…a turn New 'escalate' ladder rung between steer and block: instead of refusing or halting a stuck run, throw more intelligence at the NEXT turn and revert. Two mechanisms via doomLoop.escalation, combinable: - model: one-turn model swap (the base resolvedRequest is never mutated, so the following dispatch reverts automatically; a single- model override clears any fallback models list) - advisor: append the openrouter:advisor server tool with forwardTranscript and loop-diagnosing instructions, and pin toolChoice to it (allowed_tools/required) so the stuck model must consult before acting; object form passes through as advisor parameters A steer notice naming the detected loop accompanies the escalated turn. Budgeted: maxEscalations (default 2) per conversation, consumed at APPLICATION time (verdicts the engine never applies do not spend), escalationsUsed persisted in ConversationState.doomLoop so resumes cannot reset it, first-verdict latch so concurrent detectors in one window escalate once. Exhausted/unconfigured escalations fall through to weaker rungs; resolve-time warnings flag rung/mechanism mismatches. DoomLoopDetected action/overrideAction enums gain 'escalate' (override without config/budget downgrades to observe). 15 new tests.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Both outstanding findings are resolved as of 165fcb7:
All 111 doom-loop tests pass locally. Re-requesting review to clear the stale CHANGES_REQUESTED state. |
Devin's fresh review (head
|
There was a problem hiding this comment.
Perry's Review
Verdict: 💬 Comments / questions
Risk: 🟡 Medium
Details
Fresh review on commit 1b48987 (previous Perry review was DISMISSED on 1723d17). The prior blocker (prototype pollution in DoomLoopMonitor.restore()) and the prior nit (__proto__ setter in resolveLoopKeyMaterial's field-list subset) are both resolved. The only new change since the last review is the fix commit 165fcb7 (applying Object.create(null) to the subset) plus two merge-from-main commits that brought in publish.yaml/changeset/CHANGELOG updates from already-merged PRs.
Previous blocker — resolved
Prototype pollution via __proto__ in DoomLoopMonitor.restore() — fixed in 4860771. The streak store is now Map<string, StreakEntry>; hostile keys from persisted JSON are inert map entries. getState() emits via Object.fromEntries (own properties, safe). A regression test locks the attack case and a legitimate __proto__-named tool (doom-loop.test.ts:919).
Previous nit — resolved
resolveLoopKeyMaterial field-list subset — fixed in 165fcb7. The subset is now Object.create(null), so the __proto__ setter is bypassed. A test verifies a declared __proto__ field is preserved as an own property with a null prototype.
One consistency nit (non-blocking)
extractServerToolIdentity (model-result.ts:128) still builds its identity object as a plain {}. If a server-tool output item carries a __proto__ key (possible from JSON-parsed API responses), identity['__proto__'] = value hits the prototype setter — the same pattern the resolveLoopKeyMaterial fix just closed. The impact is minimal (the object is temporary; canonicalizeKeyMaterial uses Object.keys which skips non-own properties, so the fingerprint is just slightly wrong, not a security hole), but Object.create(null) here would make the codebase consistent and remove the edge case entirely. Inline comment posted.
Assessment
- Architecture: well-designed throughout. Deterministic detection (pure function of the transcript), round-scoped streaks (parallel duplicates count once), graduated ladder with hook overrides, cross-port fingerprint contract (RFC 8785 JCS + SHA-256 via WebCrypto with conformance vectors), and clean state persistence semantics (streaks survive resume, stop verdict survives decision-only resumes and clears on fresh input).
- Escalation recovery: one-turn model swap (resolvedRequest never mutated, so revert is automatic), advisor forcing with pinned toolChoice, budget consumed at application time and persisted. Thoroughly tested (15 tests).
- Run-level cancellation: properly composes run signal + caller signal + per-request timeout via
AbortSignal.anyper dispatch. Fresh timeout per request (not cached). Pre-aborted signal fails before network I/O. E2E tests validate against a live transport. - Test coverage: 93+ doom-loop tests, 15 escalation tests, cancellation tests (unit + e2e), integration and remediation suites. All use scripted LLM responses driving the real
callModelloop with exact-turn assertions. CI is green (638 unit tests, 51 files). - All features are opt-in (
doomLoopdefaults to off;signaldefaults to off). No behavior change for existing callers. .sentrux/baseline.json: updated in the same PR (max_depth 8→11, import edges 133→306). Process nit from the previous review still applies — the structural gate runs against this new baseline so it won't flag the complexity increase the PR introduces. Ideally the baseline would be updated separately or by the gate's own post-merge hook.
| */ | ||
| function extractServerToolIdentity(item: ServerToolResultItem): Record<string, unknown> | null { | ||
| const record = item as unknown as Record<string, unknown>; | ||
| const identity: Record<string, unknown> = {}; |
There was a problem hiding this comment.
▶ Nit — use Object.create(null) for consistency.
extractServerToolIdentity builds identity as a plain {}. If a server-tool output item carries a __proto__ key (possible from JSON-parsed API responses), identity['__proto__'] = value hits the prototype setter — the same pattern resolveLoopKeyMaterial just fixed in 165fcb7.
Impact is minimal (the object is temporary; canonicalizeKeyMaterial uses Object.keys which skips non-own properties, so the fingerprint is just slightly wrong, not a security hole). But Object.create(null) here would make the codebase consistent and remove the edge case.
| const identity: Record<string, unknown> = {}; | |
| const identity: Record<string, unknown> = Object.create(null) as Record<string, unknown>; |
extractServerToolIdentity built `identity` as a plain `{}`. Server-tool
output items are JSON-parsed API responses, so a `__proto__` key arrives
as a real own property — assigning it hit the prototype setter instead of
creating an own property, and canonicalizeKeyMaterial's Object.keys walk
then skipped it.
Perry flagged this as a consistency nit with minimal impact. It is
actually a false-positive source: two server-tool calls differing ONLY in
a `__proto__`-named field fingerprint identically, so they collide into a
streak and the detector reports a doom loop that is not one.
Adds a regression test with three same-query web_search items carrying
distinct `__proto__` values, asserting no detections. With the plain `{}`
it fails with two spurious detections.
Consistent with the field-list subset in resolveLoopKeyMaterial and the
Map-backed streak store.
Re-review of
|
`advisor` is `boolean | Record<string, unknown>`, so `advisor: false` is
the documented way to switch it off. Config resolution counted any
non-undefined advisor as a mechanism, while the recovery step skips a
disabled one (`config.advisor !== undefined && config.advisor !== false`).
So `{ escalation: { advisor: false } }` resolved a working escalate rung
that, on trigger, called consumeEscalation() and applied no override at
all — burning one of the run's limited escalations and announcing recovery
while doing nothing. Note consumeEscalation() runs before the advisor
check, so the budget was spent regardless.
Now `{ advisor: false }` alone reads the same as `{}` (no rung), and a
disabled advisor is dropped from the resolved config rather than carried
through inert.
Adds three tests: advisor:false alone resolves no escalation; alongside a
real `model` it does not appear in the resolved config; advisor:true still
resolves. The first two fail without the fix.
Reported by devin on #73.
Re-review at
|
Re-review —
|
* feat(agent): doom-loop detection for the tool-execution loop Opt-in via doomLoop on callModel. Deterministic detection of runs that stop making progress: consecutive identical tool calls (per-tool fingerprint streaks over tool-declared loopKey identity, incl. repeated empty and invalid-JSON calls) and repeated text tokens (within-response block repetition + cross-step identical-text streaks). Graduated response ladder observe -> steer -> block -> stop, per-event override via the new DoomLoopDetected hook, streaks persisted in ConversationState.doomLoop across serialize/resume, and SessionEnd.reason 'doom_loop' + ModelResult.getDoomLoopVerdict() on stop. * fix(agent): doom-loop remediation — adversarial review findings Fingerprints: RFC 8785 (JCS) canonicalization + SHA-256/UTF-8 via WebCrypto replaces cyrb53/UTF-16 (cross-port contract now real; vectors in tests/vectors/doom-loop-fingerprints.json; bigint/NaN/circular/deep key material rejected with engine fallback to full-args identity). Streaks are round-scoped: N identical parallel calls in one round count once and share the round's decision (evaluations serialized in model-emission order under the async hash). loopKey is now function | field-list | false on the tool definition (declarative forms are data — serializable, MCP-transportable); undefined returns fall back with a warning instead of colliding. markMcp accepts a loopKey override. Stop verdicts seal state (synthesized halt outputs for unresolved calls — no dangling function_call 400s on resume), gate the allow-final-response and approval-resume request paths, persist across decision-only resumes, and clear on fresh conversational turns. Text-only no-tools stops report SessionEnd reason doom_loop. Steer guidance queued before a pause persists and delivers on resume. Server tools fingerprint at the step checkpoint (observe/steer/stop). Ladder configs warn on dead rungs and block-without-stop. Documented, test-locked misses: nonce-varying args without loopKey, paraphrased text. * feat(mcp): loopKey support for MCP-wrapped tools Client-side: a loopKeys map on createMCPTools/rehydrateMCPTools (keyed by unprefixed MCP name; function | field list | false), threaded through buildTools into wrapMcpTool and attached via markMcp's injection point on both regular and generator wraps. Server-side: _meta['openrouter/loopKey'] on a tool definition (field list or false; data-only — functions can't cross the wire) is captured by listToolDefs, carried on McpToolDef, and round-trips through cache snapshots (SerializedMCPToolDef.loopKey) so rehydrated tool sets keep their identities. Client config wins over the server declaration. * chore: refresh stale sentrux baseline to current main metrics The committed baseline (coupling 0.43, 133 import edges) predates the mcp package landing — origin/main itself measures 0.47 with 306 edges against it, passing only within tolerance. The doom-loop feature's real structural delta over current main is +0.012 coupling (0.47 -> 0.49, the new lib/doom-loop module's fan-in from model-result, tool-types, tool, async-params, index, and the mcp wrapper), which is within the gate's tolerance. Baseline regenerated from origin/main (61a2a9a) via sentrux v0.5.7 gate --save, matching the CI binary version. Cycle count (1, in the mcp package) and complex-function count (9) are pre-existing and unchanged by this branch. * fix(agent): run cancellation + per-request timeouts; deflake DEV-658 e2e Fix 3 (SDK): new 'signal' option on callModel — aborting stops the tool loop at the next turn boundary AND aborts the in-flight request/ stream, rejecting with the abort reason; pre-aborted signals fail before any dispatch. RequestOptions.timeoutMs now reliably bounds EACH request even when a signal is present: the SDK skips its timeoutMs wiring whenever a request carries a signal, so the engine composes {run signal, caller signal, fresh per-dispatch AbortSignal.timeout} via AbortSignal.any at every send site (initial, tool rounds, final, retry, resume). 7 new unit tests (fake-timer driven) pin the composed semantics. Fixes 1+2 (DEV-658 e2e deflake): measured healthy path was 70-80s across three GLM requests — the final turn alone burned 50-65s on a 750-1150-token reasoning burst — leaving no margin under the old 120s budget (one CI window timed out attempt AND retry back-to-back). Now: 300s budget, bounded generation (reasoning.maxTokens 512 + maxOutputTokens 1024), a shallow unanswerable-without-search prompt that preserves the leak pressure without inviting multi-hop parametric reasoning, and a 90s per-request timeoutMs so a stalled provider fails fast and the vitest retry gets a fresh draw. Verified 3/3 live passes at 10-20s each. * refactor(agent): flatten callModel option assembly below cc gate The per-option conditional-spread chain pushed callModel to cc=16 (sentrux max 15) once the signal option landed. Build the options object once and strip undefined keys — identical absent-key semantics, one loop instead of thirteen branches. * test(agent): e2e coverage for run cancellation on the live transport The unit suite pins the signal/timeout composition against mocks; these five e2e tests prove what mocks cannot — the composed abort signal reaches the real fetch/stream. Covered: pre-aborted signal fails with zero dispatches; aborting mid-generation kills a live request fast (bounded wall-clock assertion, not exact timing); per-request timeoutMs bounds a live request even with a run signal present (the SDK-disabling configuration DEV-658 relies on); abort during tool execution stops before the follow-up dispatch; and a composed-signal run completes normally when neither bound fires. Verified 3/3 locally (~8s a run). * test(agent): harden cancellation tests against vacuous passes Unit: drop fake timers around AbortSignal.timeout — vitest fake timers cannot fake its Node-internal timer, so advancing the mocked clock proved nothing (a WRONG shared per-run timer would never fire in near-zero real time either, passing the per-dispatch test vacuously). Real tiny budgets (50-120ms, documented exception) make the two timeout tests actually discriminate; suite cost ~215ms. E2E: assert rejection IDENTITY, not just 'anything threw' — the mid-flight abort test walks the cause chain to OUR abort reason (the SDK wraps it in UnexpectedClientError), and the timeoutMs test matches the TimeoutError DOMException, so an unrelated transport/auth failure can no longer pass either test. Verified live: rejection shapes probed against the real API; 5/5 passing. * fix(agent): Map-backed streak store closes __proto__ pollution on restore Review finding (cortex): DoomLoopMonitor's streak store was a plain object; restoring a persisted blob {"tools":{"__proto__":{...}}} from client-writable state storage reassigned the store's prototype, so every unseen tool inherited the seeded streak (false stop verdicts / detection bypass). The store is now a Map — hostile keys are inert data, and a tool legitimately named __proto__ works. getState() keeps emitting a plain-JSON record via Object.fromEntries (own-property defines, no setter hits). Regression tests for both cases. Also corrects the WebCrypto doc claim (Devin): globalThis.crypto is unflagged in Node >=19, not >=18 (18 needed --experimental-global-webcrypto and is EOL); supported floor is the active LTS (CI runs Node 22). * feat(agent): doom-loop escalation recovery — advisor/model boost for a turn New 'escalate' ladder rung between steer and block: instead of refusing or halting a stuck run, throw more intelligence at the NEXT turn and revert. Two mechanisms via doomLoop.escalation, combinable: - model: one-turn model swap (the base resolvedRequest is never mutated, so the following dispatch reverts automatically; a single- model override clears any fallback models list) - advisor: append the openrouter:advisor server tool with forwardTranscript and loop-diagnosing instructions, and pin toolChoice to it (allowed_tools/required) so the stuck model must consult before acting; object form passes through as advisor parameters A steer notice naming the detected loop accompanies the escalated turn. Budgeted: maxEscalations (default 2) per conversation, consumed at APPLICATION time (verdicts the engine never applies do not spend), escalationsUsed persisted in ConversationState.doomLoop so resumes cannot reset it, first-verdict latch so concurrent detectors in one window escalate once. Exhausted/unconfigured escalations fall through to weaker rungs; resolve-time warnings flag rung/mechanism mismatches. DoomLoopDetected action/overrideAction enums gain 'escalate' (override without config/budget downgrades to observe). 15 new tests. * fix(mcp): validate cached loop keys Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(agent): preserve proto loop key fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): cover invalid advertised loop keys Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): guard loop key lookups Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): forward loopKeys through the cache-hit path `createMCPTools` forwards an explicit allowlist of options into `rehydrateMCPTools` (FORWARDED_REHYDRATE_KEYS). `loopKeys` was missing from it, so with caching enabled a client-configured loop identity was silently discarded on every cache hit — doom-loop detection went dead on warm handles only, with no error and nothing failing. `loopKeys` is declared on both CreateMCPToolsOptions and RehydrateMCPToolsOptions, and rehydrate.ts already consumes it, so the omission was the whole bug. Adds a regression test asserting the loopKey lands on the wrapped tool after a cache hit (verified to fail without the one-line fix), and a comment on the allowlist noting that anything omitted is silently dropped — the failure mode has no signal, so the next option deserves the warning. Reported by devin on #74. * fix(agent): Object.create(null) for the server-tool identity object extractServerToolIdentity built `identity` as a plain `{}`. Server-tool output items are JSON-parsed API responses, so a `__proto__` key arrives as a real own property — assigning it hit the prototype setter instead of creating an own property, and canonicalizeKeyMaterial's Object.keys walk then skipped it. Perry flagged this as a consistency nit with minimal impact. It is actually a false-positive source: two server-tool calls differing ONLY in a `__proto__`-named field fingerprint identically, so they collide into a streak and the detector reports a doom loop that is not one. Adds a regression test with three same-query web_search items carrying distinct `__proto__` values, asserting no detections. With the plain `{}` it fails with two spurious detections. Consistent with the field-list subset in resolveLoopKeyMaterial and the Map-backed streak store. * refactor(mcp): early-return guard in isSerializedToolDef Drops the duplicate isJsonSchemaObject call and, more usefully, removes a trap: the old form extracted loopKey via a ternary, so for a non-object value loopKey silently became `undefined` and hasValidLoopKey read as true. It was harmless because the conjunction re-checked the object-ness, but the local said "valid" about a value that was never inspected. Also records why each accepted loopKey shape is accepted. Suggested by Perry on #74. * fix(agent): advisor:false is an opt-out, not a recovery mechanism `advisor` is `boolean | Record<string, unknown>`, so `advisor: false` is the documented way to switch it off. Config resolution counted any non-undefined advisor as a mechanism, while the recovery step skips a disabled one (`config.advisor !== undefined && config.advisor !== false`). So `{ escalation: { advisor: false } }` resolved a working escalate rung that, on trigger, called consumeEscalation() and applied no override at all — burning one of the run's limited escalations and announcing recovery while doing nothing. Note consumeEscalation() runs before the advisor check, so the budget was spent regardless. Now `{ advisor: false }` alone reads the same as `{}` (no rung), and a disabled advisor is dropped from the resolved config rather than carried through inert. Adds three tests: advisor:false alone resolves no escalation; alongside a real `model` it does not appear in the resolved config; advisor:true still resolves. The first two fail without the fix. Reported by devin on #73. * docs(changeset): add the required API example to mcp-loop-key `.agents/skills/public-api-examples/SKILL.md` requires a fenced example in the changeset for any public-API change. `loopKeys` on createMCPTools / rehydrateMCPTools and `SerializedMCPToolDef.loopKey` are public API, and the changeset was prose-only. Shows the client-side `loopKeys` map (field-list and `false` forms, keyed by unprefixed MCP name) feeding a `doomLoop` run, plus the server-advertised `_meta['openrouter/loopKey']` alternative and the precedence rule. Shapes verified against source: loopKeys at types.ts:97, cache {store, key?} at :73, handle.tools at :112. * fix(agent): fall back when a loopKey field list collapses tool identity An empty `loopKey` field list — or one whose every field is absent from the arguments — produced an empty key subset, so every call to the tool fingerprinted identically. A detector armed to block a repeat would then refuse the SECOND unrelated call: `ls` and `rm -rf /` share an identity. `advertisedLoopKey` accepts `[]` (`[].every` is vacuously true), so once a server can advertise `_meta['openrouter/loopKey']` this is reachable from the wire, not just from a local mistake. Both shapes now warn and fall back to full arguments, mirroring how the function form already handles a degenerate result. `false` remains the way to exempt a tool. refactor(mcp): keep toCreateOptions under the complexity ceiling Ports the key-list + copy-loop from 6a87464 on the dual-protocol branch. `toCreateOptions` was at the max_cc=15 ceiling and this PR's `loopKeys` spread pushed it over, failing the structural gate. Whichever of the two PRs lands second inherits the other's key list. --------- Co-authored-by: cortex-github-agent[bot] <305999463+cortex-github-agent[bot]@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Summary
Doom-loop detection baked into the core tool-execution loop, opt-in and configurable via a new
doomLoopoption oncallModel. Design doc:agent-monorepo/planning/doom-loop-detector-design.md; this PR ships the shared machinery + the fingerprint detector (Option A) + the text detectors (Option B) + server-tool coverage, remediated against a 3-reviewer adversarial pass (agent-monorepo/planning/doom-loop-remediation-plan.md— all blockers and defects addressed; commit 2 is the remediation).A doom loop = a run that stops making progress while continuing to spend. Detected shapes:
{}calls and repeated invalid-JSON calls (which never reachexecute).web_search_calletc.) — detected post-execution at the step checkpoint.How it works
Tool-declared call identity (
loopKey— a function or a variable)Tools declare what identifies a call on their definition; the engine owns canonicalization + hashing:
Function form may return
nullto exempt an individual call.undefined, throws, and unhashable material (bigint, circular, >64 deep) fall back to the full-arguments identity with a warning — detection never fails a run. NoloopKey⇒ full validated arguments. MCP-wrapped tools:markMcp(tool, { loopKey }).Cross-port fingerprints (the contract is now real)
sha256(utf8(toolName + '\n' + jcs(keyMaterial)))— RFC 8785 (JCS) canonicalization, SHA-256 over UTF-8 bytes via WebCrypto (Node ≥18/Bun/Deno/workers/browsers, zero dependencies). Ports MUST use an RFC 8785 implementation (pipjcs,cyberphone/json-canonicalization), not stdlib JSON. Conformance vectors:tests/vectors/doom-loop-fingerprints.json(non-ASCII/emoji,-0,1e21, key-order permutations, lone surrogates, rejected-input catalog).Round-scoped streaks
A streak measures the model re-issuing a call after seeing its result — one model round trip = one piece of evidence. N identical calls fanned out in parallel within one round count once and share that round's decision (fixes the review's headline false positive). Evaluations are serialized in model-emission order under the async hash, so verdicts remain a pure function of the transcript. Interleaved calls to other tools don't reset a tool's streak; a different fingerprint for the same tool does.
Graduated ladder (defaults
observe@2, block@3, stop@6)observeDoomLoopDetectedhook (per-eventoverrideAction, last handler wins)steerdoomLoop.pendingSteer) and delivers on resumeblockstopfunction_call(no 400s on resume).SessionEnd.reason: 'doom_loop'on every stop path, including text-only no-tools runs. Gates theallowFinalResponseturn and the approval-resume unsent-results requestResolve-time config warnings: dead rungs (weaker threshold ≥ enabled stronger one) and
blockwithstop: false(unbounded block/re-issue, bounded only bystopWhen).Persistence semantics
Detector state lives in
ConversationState.doomLoop(plain bounded JSON): streaks survive serialize → resume when the resuming call passesdoomLoopagain; astopverdict survives decision-only resumes (approveToolCalls/rejectToolCalls) and is cleared by a fresh conversational turn (streaks kept — renewed repetition re-condemns quickly); queued steer guidance survives pauses.Documented, test-locked limits
loopKeycloses this per tool; the structural fix is the design doc's Option C progress ledger (follow-up PR-C). A negative test locks the miss so it's visible, not silent.Tests (all simulate the LLM — no live model)
93 doom-loop tests across three files; scripted
betaResponsesSendresponses drive the realcallModelloop; verdicts assert exact firing turns.doom-loop.test.ts(57): JCS canonicalization (key order,-0,1e21, undefined-dropping, bigint/NaN/circular/depth rejection), cross-port vector conformance, loopKey resolution (all four forms + undefined/throw fallbacks), text repetition (incl. paraphrase-miss and multi-MB perf-bound input), ladder + config warnings (dead rungs, block-without-stop), round-scoped monitor semantics (parallel duplicates, interleave, resume round-trip, corrupt-blob tolerance), server-tool detector labeling.doom-loop-integration.test.ts(16): the original scenario matrix — identical calls (observe@2/block@3, stop halts at the exact turn,SessionEnd.reason: 'doom_loop'), repeated empty calls, repeated invalid JSON, degenerate text, loopKey contracts, hook overrides (de-escalate/escalate), state persistence.doom-loop-remediation.test.ts(20, new): review-finding regression suite — B3 parallel fan-out executes fully with zero detections / blocked duplicates share one decision; B1 text-stop with tool calls leaves well-formed history + resumed request input validated; D7 text-only stop reportsdoom_loop; D1 no final-response request after a stop; D2/D3 approval-resume gating, verdict persistence across decision resumes, clearing on fresh input; D4 in-loop steer injection + steer persisted at HITL pause; D5/API1 field-list identity,falseexemption, undefined-fallback; H1 nonce evasion locked as a documented miss + caught with a field-list loopKey; H2 server-tool streaks stop the run / distinct queries don't; ON→OFF/OFF→ON config lifecycle.Verification
pnpm test: 49 files, 614 passedpnpm typecheck,pnpm lint,pnpm build: cleanSurface changes
callModeloptiondoomLoop?: boolean | DoomLoopConfig(client-only; stripped from API requests on both resolve paths)tool()config fieldloopKey?: ToolLoopKey<TInput>= function |readonly string[]|false(all tool kinds);markMcp(tool, { loopKey? })DoomLoopDetected;SessionEnd.reason+'doom_loop'ConversationState.doomLoop?: DoomLoopSerializedState(streaks +stopVerdict+pendingSteer; additive within state v1)ModelResult.getDoomLoopVerdict();@openrouter/agent/doom-loopsubpath (DoomLoopMonitor,fingerprintToolCall,canonicalizeKeyMaterial,resolveLoopKeyMaterial,detectTextRepetition, …)tests/vectors/doom-loop-fingerprints.jsonFollow-ups (tracked in the design doc)
loopOutputKey@openrouter/mcpstacked PR: per-toolloopKeysmap in the wrapper config +_meta['openrouter/loopKey']field-list transport (themarkMcpinjection point ships here)Escalation recovery (added on team request)
New
escalateladder rung betweensteerandblock: on detection, unblock the run by throwing more intelligence at the next turn instead of refusing or halting — then revert.modelslist cleared for that turn so the override can't be shadowed).openrouter:advisorserver tool withforwardTranscript: true+ loop-diagnosing instructions, and pinstoolChoice(allowed_tools/required) so the stuck model must consult before acting. Object form passes through as advisor parameters.maxEscalations(default 2), consumed at application time,escalationsUsedpersisted inConversationState.doomLoop(resumes can't reset it), first-verdict latch (concurrent detectors escalate once). Exhausted/unconfigured → falls through to weaker rungs; resolve-time warnings flag rung/mechanism mismatches.DoomLoopDetectedaction/overrideActionenums gain'escalate'; an override without config/budget downgrades toobserve.15 new tests (
doom-loop-escalation.test.ts): config resolution + warnings, ladder placement and budget fall-through, one-turn swap + revert asserted per dispatch, advisor injection/pinning/revert, parameter merging, steer notice, application-time budget consumption + persistence, hook-override honor/downgrade, text-detector escalation, double-spend prevention. Totals: 638 unit tests across 51 files.