Reduce stream trim scan cost - #1137
Conversation
|
@codex review |
|
Warning Review limit reached
Next review available in: 44 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (9)
📝 WalkthroughWalkthroughストリームメタにトリムカーソルを追加し、XADD/XTRIMによる更新、読み取り時のスキャン開始位置、TTL・バックアップの48バイト形式を対応しました。Lua fast-pathテストはリーダー切替時の再試行を検証します。 Changesストリームトリムカーソル
Lua fast-pathリーダー検証
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant prepareXAdd
participant buildXTrimHeadElems
participant StreamMeta
participant scanStreamRangeKVs
Client->>prepareXAdd: XADD with MAXLEN
prepareXAdd->>buildXTrimHeadElems: trim entries using StreamMeta
buildXTrimHeadElems-->>prepareXAdd: trim elements and trimmedThrough
prepareXAdd->>StreamMeta: update length and trim cursor
Client->>scanStreamRangeKVs: XRANGE/XREVRANGE
scanStreamRangeKVs->>StreamMeta: obtain cursor-based scan start
StreamMeta-->>scanStreamRangeKVs: adjusted scan start
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
@codex review Addressed the CI failure in TestDeltaCompactor_TTLInlineMigratesLegacyStreamTTL by updating the test to expect the new 48-byte StreamMeta shape and assert the trim cursor fields remain zero for legacy TTL migration. Validation:
|
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review Addressed the CI flake in TestLua_SISMEMBER_FastPathHit. The test now retries only transient NOTLEADER/leader-not-found errors and selects the currently reported leader Redis endpoint for the EVAL call, while preserving non-leader errors as normal failures. Validation:
|
![coderabbitai[bot]](https://cdn.github.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
store/stream_helpers.go (1)
18-20: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win48バイト（トリムカーソル付きストリームメタ）のサイズが3パッケージで独立に重複定義されています。
store.MarshalStreamMeta/UnmarshalStreamMetaが定義する48バイトのバイナリ形式に対して、adapterとinternal/backupがそれぞれ独自の同値定数を保持しており、単一のソース・オブ・トゥルースがありません。現時点では値は一致していますが、StreamMetaにフィールドが追加された場合に他2箇所が追従せず、TTL判定やバックアップの長さ検証が静かに不整合になるリスクがあります。
store/stream_helpers.go#L18-L20: このファイルが正典なので、streamMetaTrimBinarySizeを（必要なら他の定数と共に）エクスポートし、他パッケージから参照させる。adapter/redis_collection_ttl.go#L18-L19:redisStreamMetaTrimSizeBytesを削除し、エクスポートされたstore.StreamMetaTrimBinarySize（相当）を参照するように変更する。internal/backup/redis_stream.go#L58-L61:redisStreamMetaTrimCursorSizeを同様にエクスポート定数へ差し替える（バックアップ側は独立バイナリ仕様のドキュメントも兼ねているため、コメントで store 側の定義との対応関係を明記する）。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@store/stream_helpers.go` around lines 18 - 20, Export the canonical 48-byte stream metadata size from store/stream_helpers.go alongside streamMetaTrimBinarySize, then remove redisStreamMetaTrimSizeBytes in adapter/redis_collection_ttl.go and use the exported store constant. In internal/backup/redis_stream.go, replace redisStreamMetaTrimCursorSize with the same exported constant and document its correspondence to the store definition.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@adapter/redis_lua_collection_fastpath_test.go`:
- Around line 239-240: Update the leader-resolution helper around the
redis.NewClient return so it does not fall back to nodes[0] when no leader
information or matching raftAddress is found. Return a “leader not found” error
instead, allowing retryNotLeader to retry; preserve the existing client creation
for successfully resolved leaders.
---
Nitpick comments:
In `@store/stream_helpers.go`:
- Around line 18-20: Export the canonical 48-byte stream metadata size from
store/stream_helpers.go alongside streamMetaTrimBinarySize, then remove
redisStreamMetaTrimSizeBytes in adapter/redis_collection_ttl.go and use the
exported store constant. In internal/backup/redis_stream.go, replace
redisStreamMetaTrimCursorSize with the same exported constant and document its
correspondence to the store definition.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d23fb7cb-a3e0-424c-b9cf-6141337c24d5
📒 Files selected for processing (12)
adapter/redis_collection_ttl.goadapter/redis_compat_helpers.goadapter/redis_delta_compactor_test.goadapter/redis_lua_collection_fastpath_test.goadapter/redis_lua_context.goadapter/redis_lua_stream_delta_test.goadapter/redis_stream_cmds.gointernal/backup/encode_redis_coll.gointernal/backup/redis_stream.gointernal/backup/redis_stream_test.gostore/stream_helpers.gostore/stream_helpers_test.go
|
@codex review Added a production performance fix for the TTL inline migrator observed during Redis proxy monitoring. In sharded deployments, the migrator now scans only local leader groups with ScanGroupAt instead of letting every node issue whole-keyspace scans that proxy back to hot leaders. The keyviz coordinator wrapper now preserves that local-leader-group capability. Validation:
|
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (a36dac7):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
|
To use Codex here, create a Codex account and connect to github. |
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (5ef05d4):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
|
@codex review Addressed the current review feedback. The SISMEMBER fast-path test now returns a retryable leader-not-found error instead of falling back to an arbitrary node, and the 48-byte stream metadata size now uses the exported store constant from adapter and backup code. Validation:
|
|
To use Codex here, create a Codex account and connect to github. |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Author: bootjp
Summary
Risk
Tests
Summary by CodeRabbit