# Whisper Tiny v149: protected capture, chained Q/K/V, streamed 48 kHz replies

This optional demo preserves v148 and Japanese chat v145. Run on the board:

```sh
cd /home/aionic/work/vlm/whisper_tiny_v149
bash run_voice.sh --max-new-tokens 32
```

Press Enter to record a question. Wait for the three tones, then speak near Y02.
The speaker is unmuted at 75% and microphone capture is set to 85%. Model loading
happens before the first recording. The shared accelerator lock prevents another
demo from using the hardware at the same time. Exit with `/quit`.

## Selected path

1. Retain 300 ms after the final cue instead of discarding the beginning of speech.
   The WAV is not amplitude-trimmed or rewritten. Cue-dominated frames are excluded
   from onset timing only; this protects quiet initial consonants.
2. Keep v148's FP16 DMAP/MMAP encoder projections and resident encoder attention.
3. Chain each decoder layer's Q/K/V projections into one PSS root. The input is
   packed once and all three FP32 results are saved before scratch is reused.
4. Use a fused FP32 CPU routine for cached attention, avoiding strided NumPy setup.
   Cross-attention remains FP32 in the selected mode (`WHISPER_CROSS_MASK=0`).
5. Keep the MMAP vocabulary scan with original FP32 top-64 candidate re-scoring.
6. Feed completed chat sentences to Open JTalk while chat continues generating.
   Two synthesis jobs can run concurrently; playback stays in sentence order.
   The original voice and original 48 kHz synthesis settings are retained.
   Microphone recording never overlaps the reply queue.

The spoken sentence chunks must concatenate exactly to the generated text. No
recognition substitutions, forced Paris token, factual rewriting or canned reply
is used. Small-model chat factual accuracy is outside this conversion check.

## Live result — 23 October 2026, Japan time

Target: `パリはどこにありますか？`

Both the original Whisper model and v149 recognized `パリはどこにありますか`.
All nine tokens including EOS match. v148 replay of the same WAV gives identical
recognition tokens, and both chat runs generate identical 32-token capped replies.
The user confirmed the streamed 48 kHz voice was clear and audible.

| Measurement on the same recording | v148 replay | v149 live |
|---|---:|---:|
| Recognition | 20.991 s | 20.965 s |
| Cached Whisper decoder | 3.867 s | 3.681 s |
| Processing through playback completion | 45.143 s (sequential stage sum) | 41.563 s (measured interval) |
| Question onset through reply completion | 48.064 s (replay estimate) | 44.483 s |

Recognition speed is essentially unchanged. Cached decoding is about 5% faster.
The complete processing interval is about 8% shorter, primarily from overlapping
TTS with generation/playback. This is a latency improvement, not a faster TTS
model: summed synthesis work increased slightly because sentences are separate.
The reply reached the 32-token cap rather than EOS.

The original live logger counted a cue echo as voice onset and initially printed
45.883 s. `tools/compare_voice_runs.py` excludes strongly 880-Hz-dominated frames
in the first 300 ms **for timing only**, yielding 44.483 s. The recognition WAV
is unchanged. Voice onset/offset are amplitude estimates and playback timestamps
are ALSA wall times, not acoustic loopback measurements. Model loading is excluded.
The v148 full-turn value adds the same measured capture interval to its sequential
processing sum; it is a replay estimate, not a second live capture.

### Validation and counts

- Four chained Q/K/V controls match the separate projections bit for bit.
- Native FP32 cached attention passes NumPy/SciPy controls at key lengths 1, 8,
  52 and 1500; maximum absolute error was 5.37e-7.
- Selected saved Paris, saved greeting and live Paris tokens match the original.
- The saved greeting is still misheard by the original model; no broad word-error
  improvement is claimed. The fresh Paris question now retains its initial sound.
- Strict FP32 encoder feature tolerance still fails, as in v148.

| Dynamic PSS stages for this live recording | v148 | v149 |
|---|---:|---:|
| KP | 1,174,988 | 1,179,644 |
| DMAP | 1,970,672 | 1,970,672 |
| MMAP | 1,159,164 | 1,159,164 |
| Execution submissions | 451 | 355 |

Counts are PSS instruction stages, not cycles or DRAM bandwidth. Q/K/V chaining
reduces submissions by 21%; it does not reduce the projection arithmetic itself.
Whisper allocates 217,436,160 bytes; with chat, 570,523,648 bytes (544.09 MiB).
Actual simultaneous loading and execution passed inside the 768 MiB pool.

## Trials retained but not selected

`WHISPER_CROSS_MASK=15` enables MMAP cached cross-attention in all four layers.
K/V packing is chained directly after their projections and all six attention
heads run in one PSS submission. It eliminates repeated cache uploads, but changed
the saved Paris punctuation token. Individual layer trials also changed greeting
tokens. The selected mode therefore keeps FP32 cross-attention. The trial roots
remain in this build for investigation and account for some additional residency.

24 kHz and 16 kHz vocoder trials were faster, but the user selected the original
48 kHz voice. They are not selected in the launcher.

## Controls and reproducibility

- `JAPANESE_TTS_STREAM=0 JAPANESE_TTS_PARALLEL=0 bash run_voice.sh ...` restores
  full-reply sequential synthesis using the original voice.
- `WHISPER_CHAIN_QKV=0` restores three projection submissions per layer.
- `WHISPER_FAST_CACHED=0` restores NumPy cached attention.
- `WHISPER_CROSS_MASK=0` is the default. Nonzero masks are numerical trials.
- `run_recognize.sh --wav /absolute/question.wav --output-directory trial` runs
  saved-audio validation under the shared lock.

`tools/prepare_reference.py` runs the pinned original checkpoint on a host with
PyTorch. `tools/validate_original.py` checks every token and separately reports
feature tolerance. `tools/profile_instructions.py` expands actual PSS roots.
`tools/check_qkv.py` and `tools/check_cached_attention.py` isolate their respective
changes. `tools/compare_voice_runs.py` documents timing and verifies transcript,
chat-token and speech-text agreement. `RESULTS.json` stores the measured summary.

The encoder and cached decoder, native runtime, sources, reports and source archive
remain under `/home/aionic/work/vlm/whisper_tiny_v149`. Shared v148/v147 parameter
assets and v145 tokenizer/voice files are live dependencies; retain them. The v148
launcher and models were not replaced. CPU work still includes spectral features,
im2col, norms/GELU, cached attention, KV control, FP32 candidate correction and TTS.