Qwen3.8-Flash-Next on Strix Halo: a 177B Qwen4 preview at 22 tok/s, one day after release

Qwen3.8-Flash-Next is Alibaba's preview of the Qwen4 architecture, released 2026-08-26. One day later it was generating correct code on the Strix Halo box at 22.3 tok/s, before mainline llama.cpp even supports the architecture. That pipeline (Qwen ships a new architecture, unsloth has a llama.cpp branch and quants within hours, kyuz0 publishes a gfx1151 container built from the unmerged PR head, and a 177B model runs on an iGPU the next day) is half the story of this post. The other half is that this architecture is aimed, more directly than anything the box has run, at exactly the constraint that limits it.

The model

The headline numbers first, from the model card and labeled as vendor claims: at max reasoning effort it beats Claude Opus 4.6 on LiveCodeBench v6 (91.9) and SWE-bench Multilingual (81.0), trained on roughly one-ninth the compute of Qwen3.7-Plus. No independent confirmation exists yet, and Simon Willison's early low-quant tests found it weaker than Qwen3.8-27B, so let the dust settle.

The parameter count needs a sentence, because sources quote both "125B" and "177B" and both are right. The transformer backbone is 125B (512 experts, 10 routed plus 1 shared active, ~6B active per token). On top of that sit a 51B n-gram embedding table and a 4B multi-token-prediction head, which lands the on-disk total at 176.9B, confirmed by what /v1/models reports on this box. Context is 262K native, extensible to 1M. The license is qwen-community-1.0, not Apache 2.0: an MIT-style permissive grant with a clause that only bites past 100 million monthly active users.

Why this architecture matters on this hardware

The blog's long-running complaint about big models on unified memory is not decode speed, it is what long context costs: KV cache that eats the memory ceiling, and prefill that collapses as context grows. Flash-Next's design attacks both. 36 of its 48 layers are Gated DeltaNet linear attention, which carries a fixed-size recurrent state instead of a KV cache that grows with context. Only the remaining 12 layers do global retrieval, and those use Qwen Sparse Attention, selecting context at micro-block granularity rather than attending densely. Qwen's own claims for the 1M-token regime: 7.6x prefill and 4.9x decode speedup over their previous generation. Vendor numbers, but the direction is the one this class of hardware needs.

The n-gram table has a practical local-inference consequence too: it is designed to be offloadable to host RAM with async prefetch, and unsloth's quants deliberately keep it at 4-bit minimum because its random access pattern punishes aggressive quantization.

Getting it running

Mainline llama.cpp does not support the qwen4exp architecture. Support lives in PR #27742, opened by unsloth's danielhanchen hours after release (a parallel PR from Qwen's own engineer was closed in its favor, amicably, with the useful parts being reconciled), with llama.cpp maintainers pushing commits directly into the branch. kyuz0/amd-strix-halo-toolboxes, the project this box's whole serving setup is built on, publishes an experimental container built from that PR head. ROCm only; no Vulkan build of this exists yet, so for once there is no backend comparison to run.

The rest was routine: pull the container, download unsloth's UD-Q3_K_XL (83.8 GiB, three-part GGUF), launch with the flags the PR's known bugs dictate. Load took 57 seconds, mmap-backed, zero errors.

The numbers

llama-bench, single-stream, f16 KV, 3 repetitions:

Batched decode scales: 22.1 tok/s aggregate at batch 1 rises to 56.5 at batch 4 (2.6x), overall throughput from 91.7 to 183.6 tok/s. Treat those batch numbers as the raw decode-kernel path, not validated serving throughput; multi-slot serving is one of the PR's open bugs, so the server ran --parallel 1.

Where that lands among the big models measured on this same box, all first-party numbers from earlier posts:

The comparison worth staring at is prefill. DeepSeek-V4-Flash, the previous biggest model on the box, prefilled at 103 tok/s at 4K context and collapsed from there, which is what disqualified it for agentic use. Flash-Next is 177B total and prefills at 341, within 10% of a model half its total size, on an unmerged, untuned PR build. If the linear-attention design delivers its claimed shallow degradation at long context, this is the first frontier-scale model whose long-context story might actually work on this hardware. That is a conditional, not a finding; the long-context sweep is the obvious next bench.

The smoke test behaved: correct working Fibonacci code, clean stop, and at --reasoning-effort low a 530-character reasoning trace rather than the runaway chains the community reports at the xhigh default. That default is worth knowing about: the effort tiers are injected system-prompt instructions, community reports on this model family describe 15-40K-token thinking runs on trivial prompts at xhigh, and llama.cpp's --reasoning-budget is the blunt fix.

Stability, and a note for regular readers

Zero GPU faults across the whole session: load, generation, llama-bench, batched-bench through batch 4. No amdgpu queue evictions, no kernel log events attributable to the model. Regular readers will remember that a Qwen3.8-family model hard-crashed this box's GPU driver twice three weeks ago. Different model, and critically a different backend: those crashes were Vulkan RADV, this ran ROCm, consistent with the family's Vulkan crash reports being a backend problem rather than an architecture problem. Same lesson as before: backends are per model family.

What doesn't work yet

Pre-merge software, and it shows. The constraint list, each one a documented PR bug rather than a preference: quantized KV cache crashes the sparse-attention path, so f16 KV is mandatory (an echo of the DeepSeek post's identical constraint); auto-offload crashes, so -fit off; multi-slot serving desyncs, so --parallel 1; tensor-split placement fails. None of this is production material, and this post makes no production claim. It ran, it produced correct output, it survived a bench session without a driver fault. That is what "day one on a preview architecture" buys, and honestly, it is more than it bought a year ago.

The MTP head, and a standing warning

Flash-Next ships a 4B MTP head, trained multi-step, exactly the speculative-decoding accelerator that made Ornith 35% faster on this box. The PR has not wired it up yet. When it lands, a caution that this blog is now in a position to state precisely: this is a hybrid Gated DeltaNet model, and our cross-engine measurements found MTP silently corrupting output correctness on exactly this architecture class in both llama.cpp and vLLM, with GDN recurrent-state rollback on draft rejection as the identified mechanism and an open, unmerged vLLM fix. The throughput win will be real and the acceptance rates will look fine. Verify output invariance against tests that execute before trusting it anywhere that matters.

The takeaway

Twelve months ago, running a brand-new frontier-lab architecture locally meant waiting weeks for mainline support. This model went from release to generating code on an integrated GPU in about a day, through a pipeline (vendor, quant maker, container maintainer) in which no step waited for the previous one to be official. The model itself is a preview and should be treated as one. What it previews is the interesting part: Qwen4's bet is that attention should mostly be linear and sparse, which is precisely the bet that makes 256K-1M context plausible on a 128 GB unified-memory box. The next generation of frontier models may fit this hardware better than the current one does. The long-context bench will say.

Reproducibility: AMD Ryzen AI MAX+ 395 (Radeon 8060S, gfx1151), 128 GB unified memory, ROCm. Container kyuz0/amd-strix-halo-toolboxes:rocm-7.14-qwen-3.8-flash-next, built from unslothai/llama.cpp:qwen4exp/qwen3.8-flash-next (PR #27742 head, build 10678, commit b8bdf73bb). Model unsloth/Qwen3.8-Flash-Next-GGUF UD-Q3_K_XL. Flags: -ngl 999 -fa 1 -ctk f16 -ctv f16 -fit off --parallel 1 --reasoning-effort low --jinja. Bench: llama-bench -p 512,4096 -n 128 -r 3; llama-batched-bench -npp 512 -ntg 128 -npl 1,2,4. Base stack in the setup guide.