Skip to content

fix(dig-node): sweep modules-cache after every read-path §21 sync attempt - #170

Merged
MichaelTaylor3d merged 3 commits into
mainfrom
loop/2041-backfill-cache-sweep/fix
Aug 3, 2026
Merged

fix(dig-node): sweep modules-cache after every read-path §21 sync attempt#170
MichaelTaylor3d merged 3 commits into
mainfrom
loop/2041-backfill-cache-sweep/fix

Conversation

@MichaelTaylor3d

@MichaelTaylor3d MichaelTaylor3d commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Closes dig_ecosystem#2041. Defense-in-depth follow-up flagged by the #1934 PR-3 loop-security re-audit.

Why

The tier-aware <cache>/modules size-cap eviction fires after the tier-0 land + after cache_fetch_and_cache, but the read-path §21 whole-store sync did NOT sweep: sync_module_from (lib.rs:1507) lands via write_atomic (:1594) and only bumps telemetry. So with tier-0 disabled AND zero cache.fetchAndCache traffic, a remotely-triggered read-path §21 backfill could grow <cache>/modules unbounded. The capsule_store.rs comment claiming the §21 backfill kept whole-capsule storage bounded overstated.

What changed

  • Node::sync_module_and_bound (lib.rs:1476) — calls sync_module, then unconditionally runs self.evict_modules_if_needed().await after the sync attempt, and returns may_serve_locally. Wired at the read-path §21 land site dispatch.rs:768.
  • Sweep after the attempt, not on the bool — a sync whose served root differs from the requested root (the remote head advanced between resolve and sync) still lands a chain-anchored capsule under the served root via write_atomic while sync_module returns false. Gating the sweep on the true return would leave that land unswept (remotely drivable at will with the pin off + an identity configured), reopening the residual. Sweeping after every attempt closes it; the extra bounded, idempotent dir-scan on the no-land path is cheap.
  • Async evict variantdispatch.rs:768 holds NO cache_lock (verified), so evict_modules_if_needed (takes the lock fresh) is correct; evict_modules_locked would be wrong. sync_module_from is deliberately left unswept — it's also reached by cache_fetch_and_cache while holding cache_lock, so sweeping there would double-sweep / risk lock inversion.
  • maybe_backfill_capsule already covered — it → spawn_capsule_backfillgap_fill_generationcache_fetch_and_cache (capsule_store.rs:218), which sweeps at :262 under the held lock. No change needed (confirmed by trace).
  • Corrected the overstated capsule_store.rs comment + added SPEC §3.4 (the tier-aware whole-capsule bound + loop-independence across all on-demand land paths, including the served≠requested sync attempt).

How verified (TDD)

  • read_path_sync_module_land_bounds_modules_cache_without_tier0_loop — served == requested: drives the full real sync against a live mock upstream (chain-anchored capsule) with a pre-existing oversized tier-0 module, tiny cap, NO tier-0 loop / no fetchAndCache; asserts the demanded module survives and the oversized one is evicted.
  • read_path_sync_land_under_served_root_bounds_cache_even_when_served_ne_requestedthe served≠requested edge: mock §21-clone serves served=0xAA… for requested=0xBB…, so a chain-anchored capsule lands under the served root while sync_module returns false; asserts the served-root capsule landed AND the oversized tier-0 module is still evicted. Non-vacuous: re-gating the sweep on the bool makes this test FAIL while the served==requested test still passes.
  • cargo test -p dig-node-core --lib read_path_sync → 2 passed, 0 failed. fmt clean, build clean.

Follow-up (out of scope, filed)

The reshare-warm land path (module_reshare.rs promote_into_cache) also lands a whole capsule into <cache>/modules with no size-cap sweep — pre-existing, self-driven (the reshare flywheel, not a direct remote read), non-gating. Tracked as dig_ecosystem#2053.

Version

root [workspace.package].version 0.93.1 → 0.93.2 (patch, fix:); Cargo.lock regenerated.


Generated by Claude Code

claude added 3 commits August 3, 2026 13:26
Co-Authored-By: Claude <noreply@anthropic.com>
…e bound is loop-independent

The read-path §21 whole-store sync (dispatch `sync_module`) landed a whole
capsule via `write_atomic` without running the tier-aware size-cap sweep, unlike
the tier-0 precache loop and `cache.fetchAndCache`. With tier-0 disabled and no
`cache.fetchAndCache` traffic, remotely-triggered read-path backfill could grow
`<cache>/modules` unbounded (#2041 defense-in-depth).

Add `Node::sync_module_and_bound` (sync then `evict_modules_if_needed` on success)
and call it at the read-path land site. The call site holds no `cache_lock`, so the
async variant is correct; the `sync_module_from` choke-point is left unswept (it is
also reached by `cache_fetch_and_cache` under the lock). Correct the overstated
`capsule_store` comment and the SPEC modules-cache bound.

Closes #2041

Co-Authored-By: Claude <noreply@anthropic.com>
…empt, not only on success

A read-path §21 sync whose served root differs from the requested one (the
upstream head advanced) returns false from `sync_module` yet still lands a
whole chain-anchored capsule under the served root via `write_atomic`. Gating
the sweep on the true return let repeated head-advance lands grow
`<cache>/modules` unbounded — the residual the reviewer flagged.

Make `sync_module_and_bound` sweep after every sync ATTEMPT (the scan is
bounded + idempotent, so the `false`/`Err` no-land path is a cheap no-op),
add a served≠requested regression test, and tighten the SPEC wording.

Refs #2041

Co-Authored-By: Claude <noreply@anthropic.com>
@MichaelTaylor3d MichaelTaylor3d changed the title fix(dig-node): sweep modules-cache after read-path §21 backfill so the bound is loop-independent fix(dig-node): sweep modules-cache after every read-path §21 sync attempt Aug 3, 2026
@MichaelTaylor3d
MichaelTaylor3d merged commit 3209c59 into main Aug 3, 2026
16 checks passed
@MichaelTaylor3d
MichaelTaylor3d deleted the loop/2041-backfill-cache-sweep/fix branch August 3, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants