Qwen3.8-Flash-Next on Strix Halo, part 2: the missing speed was already merged upstream

The day-one post established that Qwen3.8-Flash-Next runs on this box. This one is the campaign that followed: tune it, build the missing backend, measure quality across quants, and find out where the speed went. The short version: production candidate is IQ3_XXS at -ub 2048 --parallel 2 on a fresh master build, and the biggest single performance finding cost zero lines of code, because it had already been merged upstream by people who never heard of this machine.

Tuning: two levers paid, two didn't

Method as in the tuning post: one variable at a time, 3-repeat noise floor, journaled runs, via strix-halo-optimize.

• -ub 2048: +13-28% prefill on every quant at every depth, decode flat, plateaus past 2048. (On this model the sweet spot is 2048, not the 1024 that won on Qwen3.6; calibrate per model.)
• --parallel 2: +65% aggregate decode across two streams. The multi-slot desync bug from the day-one post is confirmed fixed.
• PLE/tensor placement flags: no-op. On 128 GB unified memory everything already fits at -ngl 999; these levers exist for VRAM-constrained discrete GPUs.
• mmap: a 20-30% regression. --no-mmap stays mandatory on this box, this model included.

One measurement reframed the whole campaign. At ~22 t/s decode, this model uses roughly a quarter of the memory bandwidth its active parameters predict (~84 t/s theoretical), while rocm-smi shows the GPU 78-96% busy throughout. Decode here is compute-bound in the kernels, not memory-starved. That is unusual for MoE decode on this hardware, and it pointed at the kernel-profiling phase below.

The first working Vulkan build, and a quant-dependent crossover

No Vulkan build of the qwen4exp architecture existed publicly, so one got assembled: the exact source branch reverse-engineered from the git history of kyuz0's retired toolbox, combined with the standard Vulkan build recipe. Across four quants (Q3_K_XL, IQ3_XXS, Q2_K_XL, IQ1_M):

• Vulkan wins decode on every quant, +10-14%, no exceptions.
• Prefill depends on quant size: ROCm wins on the two larger quants (up to +14.5%), Vulkan on the two smaller ones.

The running series lesson (backends are per model family) gains a refinement: sometimes they are per quant, too.

Quality across quants: a cliff, not a slope

Quality was scored with to-bench, the execution-based harness that grew out of the MTP quality work: real coding prompts graded by running hand-written pytest suites against the model's code, no LLM judge anywhere, partial credit, identical seeds across configs. It is now a 52-task tool with a locked protocol, due for public release shortly.

Two false starts first, both caught by the protocol rather than luck: an initial run on the easy task set (saturates near 100% on any competent model, discriminates nothing, the same ceiling effect that fooled the MTP eval's first phase), and an initial run at the wrong reasoning setting for cross-cohort comparability. Re-run on the full 52 tasks, 5 seeds, locked protocol:

The non-monotonic row is not noise: IQ1_M beats the larger Q2_K_XL because two specific tasks fail 5 seeds out of 5 at Q2_K_XL, a reproducible quality cliff at that precision on those tasks. Same shape as the Ornith quant findings: quantization damage is per-task and uneven, not a smooth slope down the bit-width ladder.

The keeper is IQ3_XXS: statistically tied with Q3_K_XL (the intervals overlap heavily) at 8.7% smaller, with faster two-stream throughput than any larger option.

The profiling payoff: the fixes already existed

Why is decode compute-bound? An execution-graph dump (llama-eval-callback, ships in the toolbox) confirmed the op mix by name: GATED_DELTA_NET, SSM_CONV, MUL_MAT_ID. That matched llama.cpp issue #20354 almost exactly: GDN kernels running ~4x slower than expected on ROCm, never re-tuned for RDNA 3.5, reported on this same hardware family.

Then the actual finding, one layer deeper: this was already fixed. A HIP shared-memory fix for the GDN kernels (#20366) and a Vulkan GDN shader (#20334) both merged in mid-March, and qwen4exp itself merged to master on 2026-08-27, after both. Our custom build was hand-rebased from a pre-merge branch and never had a chance to combine all three. So instead of writing a patch: rebuild both backends from current master and re-measure. Vulkan gained +15-16% decode, ROCm +9% short-context prefill and +5% short-context decode. Free speed, zero patch-writing, correctness re-verified on the new build.

The lesson generalizes to anyone running day-one models: a pre-merge build ages backwards. Every week it lives, it silently misses mainline fixes, and the first debugging step for "this architecture is slow" is not a profiler, it is a rebuild.

Giving back, and two incidents

Two open llama.cpp PRs explicitly wanted testers on this GPU family, so both got built and benchmarked against the master baseline, with numbers posted upstream: one (GDN cache-write fusion) neutral within noise here, the other (a chunked-prefill kernel) a 16-24% prefill regression on this hardware, opposite the author's +30% on a different device. Negative results on the right hardware are exactly what draft PRs need.

And the incident file grew. The stale-server bug bit again in a new costume: kill $SERVER_PID on a toolbox-wrapped llama-server kills the wrapper, not the podman-exec'd server inside, which silently corrupted one eval run against a leftover model before a /v1/models check caught it, mid-run. That is the third measured-the-wrong-thing incident this series (the judge, the benchmark arms, now this), and the countermeasure is now reflexive: verify what is actually serving before trusting any result. Separately, a research subagent that was asked to investigate profiler tooling instead attached to and relaunched the live production server, and the only reason it was caught is that production state got verified directly rather than taking the agent's self-report at face value. Agents inherit the stale-server lesson too: trust state you checked, not state you were told.

Bottom line

Production candidate: IQ3_XXS, -ub 2048 --parallel 2, rebuilt from current master, Vulkan for decode-heavy use and ROCm for prefill-heavy at this quant tier. Not done yet, stated plainly: the quant sweep and the quality eval have not been re-run on the new master builds, so the numbers above are internally consistent on the old builds but pre-date the free upstream speed. That re-run is the last gate before calling this final.

Setup and history: day-one post, base stack, tuning methodology repo.