fix(rpc): serve StateReplay from the tipset trace cache + semaphore

[LesnyRumcajs]

Summary of changes

Changes introduced in this pull request:

• guard Forest from swarm of state replay executions via a semaphore and enforcing a cache. We are paying a bit for this cache in worst case scenarios, i.e., the target message being the first one, but it should pay off big time in the greater scheme of things.

┌──────────────────────┬───────────────────────┬─────────────────────────┬──────────────┐
│                      │    baseline (main)    │       fix branch        │    change    │
├──────────────────────┼───────────────────────┼─────────────────────────┼──────────────┤
│ Average latency      │ 204 ms                │ 0.91 ms                 │ ~224× faster │
├──────────────────────┼───────────────────────┼─────────────────────────┼──────────────┤
│ p50 / p95 / p99      │ 203 / 224 / 227 ms    │ 0.85 / 1.31 / 1.40 ms   │              │
├──────────────────────┼───────────────────────┼─────────────────────────┼──────────────┤
│ Throughput           │ 48.7 req/s            │ 10,444 req/s            │ ~214×        │
├──────────────────────┼───────────────────────┼─────────────────────────┼──────────────┤
│ Executions performed │ 300 (one per request) │ 1 (rest are cache hits) │              │
└──────────────────────┴───────────────────────┴─────────────────────────┴──────────────┘

Note that this cache is shared with some other methods so we're even less likely here to have a miss.

Reference issue to close (if applicable)

Closes

Other information and links

Change checklist

• I have performed a self-review of my own code,
• I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
• I have added tests that prove my fix is effective or that my feature works (if possible),
• I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Outside contributions

• This pull request is based on an issue that a maintainer has accepted (see Before Opening a Pull Request).
• I have read and agree to the CONTRIBUTING document.
• I have read and agree to the AI Policy document. I understand that failure to comply with the guidelines will lead to rejection of the pull request.

Summary by CodeRabbit

• Performance
  • Tipset replays are now served from a shared trace cache, avoiding repeated replay work for the same tipset.
  • RPC-triggered replays are concurrency-limited to prevent replay-heavy clients from starving the node.
• Configuration
  • Added FOREST_RPC_REPLAY_CONCURRENCY to set the default replay concurrency (half the available CPUs, minimum 1).
• Bug Fixes
  • Replay now returns cached results when available and fails with a clear "failed to replay" error when the requested message isn’t present.
• Tests
  • Added coverage for cached replay, concurrency limiting, and replay failure behavior.
• Documentation
  • Documented FOREST_RPC_REPLAY_CONCURRENCY in the environment variables reference.

[coderabbitai]

Walkthrough

State replay now uses shared tipset execution traces and a configurable semaphore to bound concurrent blocking replays. Tests, snapshots, documentation, changelog entries, and the Forest dictionary are updated accordingly.

Changes

State replay execution

Layer / File(s)	Summary
Replay concurrency configuration and permits src/state_manager/mod.rs, src/state_manager/execution.rs	StateManager initializes and clones a semaphore sized from CPU availability or FOREST_RPC_REPLAY_CONCURRENCY, holding permits across blocking replay work.
Trace-cache replay flow src/state_manager/execution.rs	StateManager::replay retrieves the tipset execution trace and returns the invocation matching the requested message CID, or a "failed to replay" error.
Replay validation and supporting references src/state_manager/tests.rs, src/state_manager/utils.rs, src/tool/subcommands/api_cmd/test_snapshots.txt, docs/docs/users/reference/env_variables.md, CHANGELOG.md	Tests cover cache reuse, missing messages, and permit sizing; the RPC snapshot reference and replay documentation are updated.

Dictionary metadata

Layer / File(s)	Summary
Dictionary entry update .config/forest.dic	The dictionary count increases to 293 and includes CPUs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RPCMethod
  participant StateManager
  participant TipsetTraceCache
  RPCMethod->>StateManager: request replay for tipset and message CID
  StateManager->>TipsetTraceCache: fetch execution trace
  TipsetTraceCache-->>StateManager: return traced invocations
  StateManager->>StateManager: select matching ApiInvocResult
  StateManager-->>RPCMethod: return replay result

Loading

Possibly related PRs

• ChainSafe/forest#7157: Both changes route tipset tracing through shared execution-trace caching.

Suggested labels: Snapshot

Suggested reviewers: eclesiomelojunior, akaladarshi, hanabi1224

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main RPC change: StateReplay now uses the tipset trace cache and is concurrency-limited by a semaphore.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch worktree-replay-cache

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch worktree-replay-cache

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 93.65079% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.18%. Comparing base (910cbf1) to head (0bd3c05).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/state_manager/execution.rs	77.77%	2 Missing and 2 partials ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
src/state_manager/mod.rs	63.34% <100.00%> (+3.00%)	⬆️
src/state_manager/utils.rs	80.81% <100.00%> (+1.19%)	⬆️
src/state_manager/execution.rs	34.09% <77.77%> (-2.42%)	⬇️

... and 17 files with indirect coverage changes

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 910cbf1...0bd3c05. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/docs/users/reference/env_variables.md`:
- Line 24: Update the FOREST_RPC_REPLAY_CONCURRENCY documentation to clarify
that a full tipset VM execution occurs only when the requested replay is not
already available in trace_cache; avoid implying that every request performs a
full execution.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: b67832ec-5469-4292-9d64-7b619763d969

📥 Commits

Reviewing files that changed from the base of the PR and between c67d8dc and 0bd3c05.

📒 Files selected for processing (8)

• .config/forest.dic
• CHANGELOG.md
• docs/docs/users/reference/env_variables.md
• src/state_manager/execution.rs
• src/state_manager/mod.rs
• src/state_manager/tests.rs
• src/state_manager/utils.rs
• src/tool/subcommands/api_cmd/test_snapshots.txt

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• filecoin-project/lotus (manual)

🚧 Files skipped from review as they are similar to previous changes (6)

• .config/forest.dic
• src/tool/subcommands/api_cmd/test_snapshots.txt
• CHANGELOG.md
• src/state_manager/tests.rs
• src/state_manager/mod.rs
• src/state_manager/execution.rs