perf(encoder): gated q/k/v+fc1 int8 (FW_ENC_INT8_ATTN_IN) — 1.23x encoder, proper-noun-safe

Extends the fc1-only int8 lever to the attention INPUT projections (q/k/v), which
feed the softmax scores (error-robust; decode runs qkv int8 default-on). Keeps
attn_out + fc2 on f32. New enc_int8_attn_in() gate (FW_ENC_INT8_ATTN_IN), default
OFF = byte-identical (conformance GREEN).

Validated on track01_16k.wav (the ledger's proper-noun clip, found on-box):
q/k/v+fc1 int8 PRESERVES FrankenSearch/Franken projects/Franco (the exact nouns
prior whole-encoder-int8 mangled to 'Frank at') with only 7 word-diffs vs f32 --
FEWER than fc1-only's 19. So the whole-attention proper-noun failure was attn_out/fc2
(residual-feeding), NOT the input projections.

MEASURED (e2e_probe large-v3-turbo, PERF_SPANS jfk x3 encoder_window window-1, min of 3, 32t):
  f32           2482.4 ms  1.00x
  fc1-only      2218.4 ms  1.119x  (19 track01 word-diffs)
  q/k/v+fc1     2015.8 ms  1.231x  (7 track01 word-diffs)   <- strictly better
Encoder ~76-82% e2e => ~1.15x e2e. jfk ts+no_ts byte-identical.

Also corrects the fc1 entry: on track01 (not on-box last round) fc1-only is 19
word-diffs, not byte-identical -- both fc1 and attn_in are mild quality tradeoffs,
proper-noun-safe, correctly gated default-off. Ratio vs ORIG unchanged default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: Dicklesworthstone

docs/NEGATIVE_EVIDENCE.md

@@ -3,6 +3,22 @@
 This ledger records blocked, neutral, rejected, or non-comparable performance
 evidence. It exists to prevent stale optimism from being reused as proof.
 
+---
+## 2026-07-07 - AshHeron: LAND (gated WIN, measured on the PROPER-NOUN clip) — **ENCODER attention-INPUT (q/k/v) + fc1 int8 (`FW_ENC_INT8_ATTN_IN`, default OFF = byte-identical) = 1.231× encoder_window, and it PRESERVES the proper nouns on track01 (the exact "FrankenSearch" clip the prior whole-encoder-int8 digs mangled to "Frank at"). This ISOLATES the encoder-int8 culprit: the proper-noun failure was NOT the q/k/v input projections (they feed softmax scores → error-robust) — it was `attn.out`/`fc2` (residual-feeding). Adds ~19% of the encoder (qkv_proj) on top of the fc1 lever for ~2× the speedup at equal proper-noun fidelity.**
+
+**Land-or-dig result: found track01_16k.wav on-box (`/data/tmp/blackthrush-targets/`, the ledger's proper-noun clip) and used it to (a) properly validate the just-landed fc1 lever and (b) extend int8 to the attention INPUT projections.** AGENT_NAME=AshHeron. The prior encoder-int8 rejections (812/1138) co-quantized ALL of attention + fc2 and hit an intrinsic proper-noun error; they never isolated which GEMM. The decode side runs qkv int8 default-on because q/k/v feed the SCORES→softmax (error-robust), unlike out/fc2 which feed the residual. New `enc_int8_attn_in()` gate (`FW_ENC_INT8_ATTN_IN`) i7-quantizes attn_q/k/v + mlp_fc, leaving attn_out + mlp_proj/fc2 on f32. src: `src/native_engine/{encoder.rs,mod.rs}`. Default OFF ⇒ all i7 None ⇒ byte-identical (conformance GREEN by construction; the OFF f32 path is untouched).
+
+**MEASURED (`e2e_probe large-v3-turbo`, no_ts; SPEED = jfk×3 `encoder_window` window-1, min of 3, 32t; QUALITY = track01_16k.wav 124.5s ~5 windows, f32-vs-int8 word diff):**
+```
+config                     encoder_window   ratio   track01 word-diffs   proper nouns (FrankenSearch/Franco)
+f32 (OFF, default)         2482.4 ms        1.00×   —                    ✓ (baseline)
+fc1-only (FW_ENC_INT8_FC1) 2218.4 ms        1.119×  19 / ~250 words      ✓ preserved
+q/k/v+fc1 (ATTN_IN)        2015.8 ms        1.231×  7 / ~250 words       ✓ preserved
+```
+attn_in is STRICTLY BETTER than fc1-only: FASTER (1.231× vs 1.119× encoder) AND cleaner on track01 (7 vs 19 word-diffs — both proper-noun-safe; the diffs are ambiguous filler like "it's"/"you know"/punctuation, non-monotonic in quant amount at the int8 noise floor). Encoder is ~76-82% e2e ⇒ **~1.15× e2e** on encoder-dominated workloads. jfk×3 ts + no_ts are byte-identical for both.
+
+**CULPRIT ISOLATED — refines "encoder int8 quality-fatal".** The whole-attention "Frank at" failure (1104/1132/1138) is NOT the q/k/v input projections — those are softmax-error-robust and preserve "FrankenSearch" perfectly. It is the RESIDUAL-feeding `attn.out`/`fc2` (same class as decode fc2, which needed block-wise + stayed gated, [[project_int8_mlp_fc1_default_on]]). So the encoder int8-SAFE set is {q, k, v, fc1} (softmax/GELU-absorbed) and the UNSAFE set is {attn_out, fc2} (residual). **CORRECTION to the fc1 entry below (2026-07-07):** that entry said fc1-only is "byte-identical on every on-box clip" — true for jfk, but on track01 (not on-box then, now is) fc1-only produces 19 word-diffs (proper nouns preserved). Both fc1 and attn_in are mild quality tradeoffs on hard audio, not byte-exact — hence correctly gated default-off; the "byte-identical" claim held only for the easy jfk clip. Still promote-candidates pending owner sign-off on the word-drift, but NOT default-flip byte-exact. Gated default-OFF (non-byte-exact int8): ratio vs ORIG UNCHANGED default (byte-identical off); flag ON ⇒ 1.231× encoder. A/B: `FW_ENC_INT8_ATTN_IN=1` vs unset on `e2e_probe` (speed: PERF_SPANS jfk; quality: track01_16k.wav).
+
 ---
 ## 2026-07-07 - AshHeron: LAND (gated WIN, measured) — **ENCODER `mlp.0`/fc1-ONLY int8 (`FW_ENC_INT8_FC1`, default OFF = byte-identical) = 1.80× on the fc1 GEMM ⇒ 1.105× on the whole encoder_window, TRANSCRIPT BYTE-IDENTICAL to the f32 default on every on-box clip (jfk×3 ts + no_ts + test_10s_speech). This is the FIRST encoder-int8 lever that does NOT break faithfulness — because it isolates the ONE GEMM whose error GELU absorbs (fc1 feeds GELU), applying the PROVEN decode `mlp_0`-only recipe to the encoder, and keeps attention + fc2 on f32 so the cross-attention proper-noun ALIGNMENT that all prior encoder-int8 digs mangled is preserved.**
 

src/native_engine/encoder.rs

@@ -476,6 +476,19 @@ impl EncoderWeights {
                 l.mlp_fc_i7 = Some(nn::quantize_mat_to_i7(&l.mlp_fc_w));
                 l.mlp_proj_i7 = Some(nn::quantize_mat_to_i7(&l.mlp_proj_w));
             });
+        } else if super::enc_int8_attn_in() {
+            // fc1 + attention INPUT projections (q/k/v) int8, keeping attn_out + fc2 f32.
+            // Q/K/V feed the attention SCORES → softmax (error-robust, the decode side
+            // runs qkv int8 default-on); attn_out + fc2 feed the RESIDUAL (not absorbed,
+            // the culprit class). Tests whether the prior whole-attention int8 proper-noun
+            // failure ([[project_turbo_encoder_dominates]]) came from the in-projections
+            // or from the residual-feeding out/fc2. `FW_ENC_INT8_ATTN_IN`, default off.
+            layers.par_iter_mut().for_each(|l| {
+                l.attn_q_i7 = Some(nn::quantize_mat_to_i7(&l.attn_q_w));
+                l.attn_k_i7 = Some(nn::quantize_mat_to_i7(&l.attn_k_w));
+                l.attn_v_i7 = Some(nn::quantize_mat_to_i7(&l.attn_v_w));
+                l.mlp_fc_i7 = Some(nn::quantize_mat_to_i7(&l.mlp_fc_w));
+            });
         } else if super::enc_int8_fc1_only() {
             // fc1-ONLY int8 (maddubs): apply the PROVEN decode `mlp_0`/fc1-only recipe
             // ([[project_int8_mlp_fc1_default_on]]) to the ENCODER MLP. Only `mlp.0`/fc1

src/native_engine/mod.rs

@@ -354,6 +354,25 @@ pub(crate) fn enc_int8_fc1_only() -> bool {
     })
 }
 
+/// Whether to int8 the encoder attention INPUT projections (q/k/v) IN ADDITION to
+/// `mlp.0`/fc1, keeping `attn.out` + `mlp.2`/fc2 on f32. `FW_ENC_INT8_ATTN_IN`,
+/// **default OFF = byte-identical**. Q/K/V feed the attention SCORES → softmax
+/// (error-robust; the DECODE runs qkv int8 default-on), while out/fc2 feed the
+/// RESIDUAL (not GELU/softmax-absorbed — the [`enc_int8_fc1_only`] culprit class).
+/// A faithfulness probe: does the prior whole-attention int8 proper-noun failure
+/// ([[project_turbo_encoder_dominates]]) come from the in-projections or from the
+/// residual-feeding out/fc2? NON-byte-exact when ON ⇒ owner-gated. Takes precedence
+/// over [`enc_int8_fc1_only`]; subordinate to full [`enc_int8_enabled`].
+pub(crate) fn enc_int8_attn_in() -> bool {
+    static ON: OnceLock<bool> = OnceLock::new();
+    *ON.get_or_init(|| {
+        matches!(
+            std::env::var("FW_ENC_INT8_ATTN_IN").ok().as_deref().map(str::trim),
+            Some("1" | "on" | "true" | "yes")
+        )
+    })
+}
+
 /// Error-feedback weight quant for the DECODER per-row int8 weights (`nn::quantize_f16_to_i8`,
 /// used by the default-ON `gemv_i8` path: logits, mlp_0, qkv, cross_q, self_out, cross_out).
 /// `FW_DEC_EF`, default OFF = byte-identical. The decoder int8 is DEFAULT-ON and diverges from