DeepSeek-V4-Flash on Strix Halo: it runs, and now we know how fast

Can the biggest, smartest model that physically fits on a Strix Halo box earn a place in the daily rotation? DeepSeek-V4-Flash at IQ2_M is 91 GB, right at the edge of what 128 GB of unified memory allows. It benchmarks like a frontier model: MMLU-Pro 86, GPQA 88, SWE-bench 79. If it ran at usable speed, it would be the best local model this hardware can hold.

So I benchmarked it. It runs. Here is how fast.

The setup

Everything below is measured on my Strix Halo box (AMD Ryzen AI MAX+ 395, 128 GB unified memory, the machine from the setup guide). Model: DeepSeek-V4-Flash, IQ2_M quant, 91 GB on disk. Flags: -fa 1, as always on this machine.

One hard constraint up front: the KV cache must be f16. Quantizing it to q8_0 produces garbage output on this architecture. That is not a tuning preference, it is mandatory, and it costs you memory headroom you do not have much of at 91 GB of weights.

ROCm vs Vulkan

Both backends load and run the model cleanly. No crashes, flash attention works. But they are not equal:

Vulkan wins generation by 33%. For a chat or agentic model, generation speed is the number that matters, so Vulkan is the backend for this model. ROCm only looked competitive on a tiny 64-token prefill; at realistic prompt sizes Vulkan pulls ahead there too.

Generation holds steady at roughly 13 t/s across context. That is below the ~19 t/s figure cited in the research around this model. My guess: that number came from a different quant or from the custom ROCmFPX pipeline, not from stock llama.cpp on gfx1151.

The prefill problem

Generation is slow but usable. Prefill is the real problem.

Prefill drops from 148 t/s at 512 tokens of context to 103 t/s at 4096. It keeps degrading from there. A full-matrix run appeared to hang on the pp16384 test. It had not hung. It was genuinely grinding along at an extrapolated ~50 t/s.

Do the arithmetic on an agentic workload. A 16K-token prompt at ~50 t/s prefill is minutes of waiting before the first output token. Every tool call that re-submits context pays that price again. My suspicion is that DeepSeek-V4's sparse-attention and indexer kernels simply are not well optimized on gfx1151 yet, so the architecture's efficiency tricks turn into overhead here.

Where it lands in the lineup

This makes DeepSeek-V4-Flash the slowest big model on the box, not the crown jewel:

Qwen3.5-122B is twice as fast, half the size on disk, and leaves real memory headroom instead of running at the ceiling. At 91 GB plus f16 KV cache, DeepSeek-V4-Flash leaves almost nothing spare, and running that close to the limit is exactly where this machine's memory-pressure failure modes live.

Honest verdict

Feasible but not practical. That is the whole finding.

DeepSeek-V4-Flash is the highest-quality model that fits on this hardware, and you pay for that quality with 13 t/s generation, prefill that collapses on long prompts, and a memory footprint that crowds out everything else. As a daily workhorse it loses to Qwen3.5-122B on every operational axis. The "big quality" default does not change.

Where it could earn a slot: a rare, load-on-demand "I need the single best answer and I will wait" mode. Short-context reasoning tasks, where its weak prefill does not bite and its benchmark-topping quality does. Wired into llama-swap as an occasional route with Vulkan and f16 KV, evicting everything else first. Not for agentic work. Not for long context. Not for anything interactive.

There is a general lesson in here for local inference on this class of hardware. Fitting is not the bar. A model can load, run cleanly, and pass every smoke test, and still be the wrong choice because the tokens-per-second economics do not work for how you actually use it. Measure generation speed, measure prefill at the prompt sizes your real workloads produce, then decide. The best model you can fit is not automatically the best model you can use.