Add candidate lookup + strategy-hash cache to topology_index#164
Draft
aminaramoon wants to merge 1 commit into
Draft
Add candidate lookup + strategy-hash cache to topology_index#164aminaramoon wants to merge 1 commit into
aminaramoon wants to merge 1 commit into
Conversation
Grow topology_index into the place that resolves and memoizes reservation candidates, and wire it into memory_reservation_manager. - reservation_request_strategy gains a virtual hash() (typeid default, kept non-pure for API compat) overridden by all six strategies to mix their fields; a file-local hash_combine helper backs them. - topology_index gains get_spaces_of(tier) (mutable + const overloads), get_candidates(strategy) memoized by strategy hash under a shared_mutex (concurrent cache hits, exclusive insert only), and a build() factory. The index keeps a non-owning back-pointer to the manager. - memory_reservation_manager holds an atomic shared_ptr<const topology_index> with set/get accessors and routes select_memory_space_and_make_reservation through the memoized candidate path, pinning the index in a local shared_ptr so a concurrent swap cannot free the candidate span mid-use. - Adds test/memory/test_topology_index.cpp (build, get_spaces_of both overloads, hash distinctions, candidate memoization, no-manager throw, end-to-end routing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aminaramoon
marked this pull request as ready for review
July 22, 2026 22:57
aminaramoon
marked this pull request as draft
July 22, 2026 22:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Grows
topology_indexinto the place that resolves and memoizes reservation candidates, and wires it intomemory_reservation_manager.reservation_request_strategy::hash()— new virtual method (typeid-based default, kept non-pure so downstream custom strategies still compile) overridden by all six strategies to mix their fields; backed by a file-localhash_combinehelper.topology_index— newget_spaces_of(tier)(mutable + const overloads),get_candidates(strategy)memoized by strategy hash under astd::shared_mutex(concurrent cache hits, exclusive lock only on insert), and abuild(topology, manager)factory. The index keeps a non-owning back-pointer to the manager (documented: the manager must outlive the index). Existingnuma_node_of/gpus_of/gpu_idsand both constructors are unchanged, soio_context/prefetching_cachekeep compiling.memory_reservation_manager— holds anstd::atomic<std::shared_ptr<const topology_index>>withset_/get_topology_index, and routesselect_memory_space_and_make_reservationthrough the memoized candidate path. The index is loaded into a localshared_ptrand pinned for the lifetime of the returned candidate span, so a concurrentset_topology_index()can't free the cache mid-use.Design notes
strategy.hash(); collisions only cost cache accuracy, not safety.Testing
New
test/memory/test_topology_index.cpp(6 cases, 26 assertions) coveringbuild(), bothget_spaces_ofoverloads,hash()distinctions, candidate memoization (cache-hit returns identical backing storage), the no-manager throw, and end-to-end reservation routing.pixi run buildlinks cleanly.[topology_index](26 assertions) and[memory_space](173 assertions) suites pass on an RTX 6000 Ada.🤖 Generated with Claude Code