LFM2.5-24B-A2B on Strix Halo: 110 tok/s, tools 8/8, and a model that will not stop thinking

LiquidAI shipped LFM2.5-24B-A2B on Monday. By Wednesday it had been through the full house benchmark on halo. Short version: it is the fastest model on the machine by a wide margin, it passed every tool-calling check, and it thinks before every answer whether you want it to or not. All three matter. The third one changes how you should read the first.

What it is

A 24.0B-parameter mixture of experts with about 2.45B active per token: 64 experts, top-4 routing, 40 layers. The unusual bit is the layer mix - 30 of the 40 are short-convolution blocks and only 10 are attention. That is the LFM2 architecture, and it has a practical consequence: only those 10 attention layers keep a KV cache, so a 131K context costs very little memory. Trained on about 33T tokens, then put through agentic RL inside real harnesses - Hermes Agent and pi among them, the two I run. 16 languages, Finnish not one of them. License is LFM Open License v1.0: commercial use only for organisations under $10M annual revenue, which is a real restriction behind the "open-weight" label.

LiquidAI's own pitch in the release post is quite specific: an on-device agent model that reasons, calls tools and runs continuously on your own hardware. They say it themselves: not for knowledge-heavy work, pair it with retrieval. Their benchmark table backs that framing: it leads Qwen3.6-35B-A3B on the agentic rows (AutomationBench 27.69 vs 8.72, LiveClaw 48.09 vs 38.19) and loses on knowledge and coding (GPQA Diamond 74.42 vs 83.69, SWE-Bench Verified 51.0 vs 66.2). Vendor figures, yes, but a vendor telling you where its model is weak is worth something.

And there is no off-switch for the reasoning. A user asked on the model card discussions for an instruct variant; the answer from Liquid was that thinking is the point for agentic use, you can pre-fill a closing think tag to skip it but it is not recommended, and if you really need a non-thinking model, fine-tune the base. Keep that in mind for later.

How I measure, and why not llama-bench

So as you probably already know, I have a Strix Halo box, halo: AMD Ryzen AI MAX+ 395, 128 GB of unified memory, llama.cpp, the usual. If not, the setup page has the details. New this time: the official Q4_K_M GGUF, 14.5 GB, the same quant LiquidAI used for their advertised Strix Halo figures, on the Vulkan backend with two private 131K slots and the model card's sampling (temp 0.1, top-k 50, repeat penalty 1.05).

House rule, written down this week after one too many misleading numbers: llama-bench is not a speed figure. It feeds the model a synthetic token stream with no chat template, no slot scheduling, no prompt cache and no reasoning. Fine as a relative screen between two backends. Useless as a claim about how fast you get answers. So the headline numbers below come from two other harnesses, both through the live serving endpoint with the production config:

• A realistic sweep: prefill on real Python source (transformers model code) at roughly 2K, 8K and 24K tokens with prompt caching off, decode on a real coding task (thread-safe LRU cache plus five pytest cases), reading the server's own timings.
• A live-serving A/B: time to first token, per-request and aggregate tokens per second, measured client-side at one and two concurrent requests, with a codeword in each prompt that must come back so cross-request mix-ups show up as failures.

Then the same two harnesses were re-run the same day on the two production models it competes with: Qwen3.6-35B-A3B at Q8_0 and Gemma-4-26B-A4B at an 8-bit Unsloth quant, both on ROCm. Same day matters: earlier numbers are context, not a comparison.

Results

Prefill on real source, tokens per second:

*Gemma's 2K row was its cold first request of the day and is not representative.

So prefill is roughly 1.5x Qwen's at every depth, and decode is 2.3x Qwen and 1.65x Gemma. On the live-serving run with one request in flight, time to first token was 0.8 s at 2K, 3.2 s at 8K and 15.9 s at 31K, with per-request decode holding 108, 104 and 91 tok/s at those depths. Qwen at the same points: 1.3 / 4.9 / 22.3 s to first token, decoding at 47, 45 and 41. Under two concurrent requests the aggregate held up fine (97 tok/s at 2K, 50 at 8K); the per-request spreads get wide, but that is structural, one request's prefill waits behind the other's, and it reproduced on the re-run.

On the mechanics that an agent harness actually cares about:

• Tool calling: 8 of 8 checks passed - single call, multi-step round trip, forced tool_choice, parallel calls, with reasoning on and with it off.
• Concurrency correctness: 18 of 18 requests got their own answer, and all 18 matched the solo run.
• Agent loop, two concurrent episodes at a time, four episodes: 3 succeeded, none got stuck, no duplicate tool calls, up to 5 calls per turn, contexts up to 53K tokens. The one failure submitted an empty answer after two calls.
• The open llama.cpp issue #30490, where this model sometimes emits tool calls as plain-text XML that the parser then silently drops, did not reproduce here. It is still open, and the failure mode is nasty, a lost call that looks like a normal stop, so test for it on your own build.

Memory is small: 14.5 GB of weights plus about 2.5 GB of KV per full 131K slot. Two full-context slots and the model together fit comfortably in a corner of the box.

A note on what I am not showing. The llama-bench screen did run, on both Vulkan and ROCm, and it suggested ROCm keeps prefill speed better at very long context. That is a synthetic hint, not a result, and ROCm has not been tested as a live route for this model yet. And LiquidAI's own advertised figure for this chip class, about 1641 prefill and 114 decode at 4K input, comes from a different method than mine, so I am not going to line it up against the table above and call it a head-to-head. The order of magnitude agrees. That is all I can honestly say.

Two caveats on the comparison itself. It is not quant-for-quant: LFM is 4-bit, the other two are 8-bit, and a 4-bit Qwen would close some of the gap. And two rows in the live-serving table were contaminated by another agent's traffic during the first pass and had to be re-run; the figures above are the re-run values.

The always-thinking problem

Here is the thing that the tokens-per-second number hides. On the decode half of the realistic sweep, the model gets a 2500-token budget to write an LRU cache and five tests. Qwen used that budget to produce 1819 characters of answer. Gemma produced 5128. LFM2.5 produced zero: all 2500 tokens were reasoning, and it was still reasoning when the budget ran out. In the live-serving harness, which asks for a codeword and then a list, it hit the 256-token limit on every single request without ever emitting the codeword. Still thinking.

That is not a bug. It is the design, and Liquid is upfront about it. But it means raw decode speed overstates time-to-answer by a factor that depends entirely on how long the model decides to deliberate. 110 tok/s is genuinely fast. 110 tok/s with 2500 tokens of preamble before the first useful character is a 23-second wait, which is slower to a usable answer than Qwen at 47 tok/s writing the answer straight away. The agent-loop test, where the reasoning is doing real work between tool calls, is where the speed actually cashes out: 4-5 turns, 11-13 tool calls, done in seconds.

I'd put it this way: this model's speed is best measured in tool calls per minute, not tokens per second. If your workload is reason, call, read, call, it is the quickest thing on this hardware. If your workload is "write me the function", budget for the thinking or pick a model that can turn it off. And no, you cannot simply cap it - next section.

So can you just cap the thinking? Tested that too

The obvious next move is a reasoning budget: llama.cpp can enforce a per-request thinking cap in the sampler and inject a "wrap it up" message when it runs out. Tried it the same afternoon, on 13 small tasks with checkable answers (dates, ages, unit conversions, a couple of tool calls) at temperature 0, plus the agent loop again.

Two things fall out of this. A budget at 2048 does nothing, because the model rarely gets there on short questions. Below that it costs correctness, and the failure mode is specific: the model does not know a budget exists, llama.cpp just cuts the thought off, and the model then commits to the unfinished thought. "Tuesday" instead of Wednesday. In the agent loop it is worse - it submits its plan ("I need to find SERVICE_SIGNING_SALT in...") as the final answer and stops.

A polite nudge works better than a cap. "Think briefly: keep your reasoning short and then answer." in the system prompt kept every single-shot answer correct and took a quarter less time. Baked into the chat template server-side so every client gets it, same thing: 13/13, half the tokens, tool gates still 8/8. So far so good... and then the agent loop fell to 1 of 4, the same way the hard budgets did. Reasoning too little, it hands in the plan instead of executing it.

Which is the whole point of the model, so the route is back on the stock template, the nudged variant kept as a reference file. I'd say: for an agent harness, leave its reasoning alone. For a client that only asks one-off questions, put the brief-reasoning line in that client's own system prompt and enjoy the speedup. Small samples - 13 tasks, 4 agent episodes per setting - so a direction, not proof. But the agent result lines up across three independent ways of shortening the thinking, and that is harder to dismiss.

Where I'd use it, and what is still untested

Where it fits: general and agent tasks on the local box. Routing, tool dispatch, summarising retrieved material, the glue work inside a harness where the model needs to be quick and reliable at calling things rather than deep. The correctness gates passed clean, the memory footprint is tiny, and two users can share it at full context without it falling over. Not for coding, by LiquidAI's own account and by the benchmark table, and I have no reason yet to argue.

What is missing is a quality evaluation. Everything above is speed and mechanics. Nothing here tells you whether its answers on general tasks are good, and the early community signal is thin and mixed: one hands-on report on an M5 Pro got ~120 tok/s from the Q4 but found tool calling in their coding harness "wasn't good", which does not match my 8/8 but was a different harness and a different quant. Anecdotes, both ways. The earlier post on the 2.6B sibling found a model that reads and executes well but knows very little and cannot tell which mode it is in; whether ten times the total parameters moves that needle is exactly the question a general-task eval needs to answer. That is the next window.

Also still to come from Liquid's side: a DSpark draft model for speculative decoding, listed as "coming soon". On a model that already decodes at 110 and spends most of its tokens thinking, a drafter that accepts well on reasoning text could be a very very interesting thing...