Skip to content

Commit 2ed6471

Browse files
perf(native_engine): hash model version tag on a background thread at load
Hotspot #4: sha256 over a 1.6GB model file cost 3.0s synchronously at raw_output build. Now warmed on a spawned thread at load; OnceLock blocks any early caller. Isomorphism: same tag value, same outputs (golden OK). Measured: version_tag span 3044ms -> 0.0ms. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent cf3ec1c commit 2ed6471

1 file changed

Lines changed: 16 additions & 0 deletions

File tree

src/native_engine/mod.rs

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -486,6 +486,22 @@ impl NativeWhisperModel {
486486
// unbounded with stale entries for models that are never reloaded.
487487
cache.retain(|_, w| w.strong_count() > 0);
488488
cache.insert(canonical, Arc::downgrade(&model));
489+
drop(guard);
490+
491+
// Warm the version tag on a background thread: the SHA-256 of a
492+
// multi-GB model file costs seconds, and every successful run needs it
493+
// for `raw_output` / replay envelopes. Hashing here overlaps with
494+
// encode/decode instead of stalling output assembly; `OnceLock`
495+
// blocks any concurrent `version_tag()` caller until the value is
496+
// ready, so observable behavior (the tag itself) is unchanged. The
497+
// clone keeps the model alive until the hash finishes (bounded by
498+
// hash time; documented tradeoff).
499+
let warm = Arc::clone(&model);
500+
let _ = std::thread::Builder::new()
501+
.name("fw-model-hash".into())
502+
.spawn(move || {
503+
let _ = warm.version_tag();
504+
});
489505
Ok(model)
490506
}
491507

0 commit comments

Comments
 (0)