AMD announced yesterday that it is acquiring Taalas, a Toronto startup that etches model weights directly into silicon. Terms undisclosed, deal expected to close in Q4. The Register's coverage calls the chips what they are: model-specific integrated circuits. Weights live in a mask-ROM fabric on the die. KV cache and LoRA-class adapters live in on-chip SRAM. No HBM, no advanced packaging, no liquid cooling.

The headline number, from Taalas's February test chip: Llama 3.1 8B at roughly 17,000 tokens per second per user. That is a vendor figure, on an old 8B model, using an aggressive custom 3-bit quantization the company concedes degrades quality (gen 2 moves to standard FP4). Discount it as much as you like. Even at a quarter of the claim it is a different universe from anything else in inference.

I think this is one of the more interesting moves AMD has made against Nvidia's position, and not for the reason the coverage leads with.

Decode is bytes per token. Taalas moves the bytes on-die.

Regular readers have watched this blog measure one lesson from three directions this summer. Decode speed is memory traffic per token. Laguna on Strix Halo: 8B active parameters versus 3B active predicted the performance gap almost exactly. Laguna on a single H100: a 118B MoE out-decodes a dense 27B on the same card because it reads 4.5 GB per token instead of 27. MoE got fast by shrinking the bytes you stream from memory.

Etched weights are the endpoint of that curve. The weights never cross a memory bus at all; they are physically part of the logic. The whole weight-streaming bottleneck, the thing that makes a ~220 GB/s Strix Halo decode at 30 t/s and an HBM-equipped H100 decode at 130, is simply not present in the architecture.

There is a second strategic layer here that I find more interesting than the benchmark. Nvidia's moat is not only CUDA. It is priority access to HBM supply and advanced-packaging capacity, the two binding constraints of the entire AI hardware industry. Taalas chips need neither. They are built on TSMC's mature 6nm process, and per Reuters, only two metal layers are customized per model, so a respin takes about 2 months instead of the ~6 a new processor needs. AMD is not just buying speed. It is buying a way to manufacture inference capacity out of parts of the supply chain nobody is fighting over.

The obvious objection, and the caveat I reached for first: you are stuck with the model you etch.

Model lock-in is already here. We just call it production.

Here is my own datapoint. I coded with GLM 4.7 from November to a few weeks ago. Nine months on one model, in a period when new models shipped monthly. At Taalas's respin cadence, that is four or five etch cycles I would not have used.

That is not laziness. The model is coupled to the harness. Prompts, tool-call conventions, stop behavior, the failure modes you have learned to route around: all of it is tuned against one model's quirks. Swapping models means re-validating the whole workflow, and the re-validation usually costs more than the newer model's marginal quality gain. Anyone running agents in production knows this. The release cadence and the adoption cadence are different clocks, and the press keeps conflating them.

So the supposed fatal flaw of etched silicon, "once deployed you're stuck with that model," is not a new constraint. It is the existing production reality made physical. You are stuck anyway. The question is whether you are stuck at 130 tokens per second or at thousands.

The topology argument goes one step further. My own setup in pi-ensemble uses a frontier-class model for the PM role and faster, cheaper models for the subagent runs. I do not think that shape is temporary; I think it is where agentic production is heading generally. And it maps directly onto the hardware split AMD is reportedly building: flexible GPUs for the tier where you might genuinely want next quarter's smarter model, etched silicon for the high-volume tier. The subagent workhorse is the ideal etch candidate: it dominates token volume, its quality bar is "sufficient," its latency compounds across every fan-out, and its model choice is stable precisely because nobody re-tunes their subagent prompts for fun. The economics of a model-specific chip want a model that is high-volume, quality-stable, and frozen. That is a subagent workhorse, described exactly.

The question that matters for this blog: does any of this reach us?

Everything above is a datacenter story. AMD reportedly plans to pair Taalas accelerators with Instinct GPUs in its Helios racks: prefill on GPUs, decode on etched silicon. Model houses and inference providers, who already keep API models live for a year or more, are the natural first customers. Fine.

But look at the bill of materials. The HC1 is a big die, reticle-sized, yet it sits on a mature 6nm process with no HBM, no advanced packaging, no exotic cooling. Those three absences are most of what makes modern AI accelerators expensive. A single-card product holding a frozen 20B-class model does not have an obvious reason to cost five figures. Whether it could retail in the hundreds of dollars is speculation, mine and nothing more, but nothing in the physics forbids it. A cheap PCIe card that decodes a known-good workhorse model at thousands of tokens per second would slot into a local setup exactly where the subagent tier lives today: PM role on an API or the iGPU, fan-out on the card.

Whether that product ever exists is now entirely AMD's decision, and that is the part of this acquisition I will actually be watching. Taalas as an independent company might eventually have sold silicon to whoever paid. Taalas inside AMD sells where AMD's margins point, and margins point at racks.

Because look at what the local-inference roadmap offers otherwise. Speed on this blog's hardware has come from exactly two kinds of lever this year. Incremental software: speculative decoding, which failed twice on our hardware (ngram net-negative, DFlash broken or unavailable), and KV-cache compression like TurboQuant, which buys capacity, not decode speed. And incremental hardware: each LPDDR generation nudges unified-memory bandwidth up a few tens of percent. Meanwhile the unified-memory thesis I keep defending is a capacity thesis. The Strix Halo box fits models a 5090 cannot hold, at 13 to 30 tokens per second, and nothing on the visible roadmap changes those decode numbers by more than increments.

Etched silicon is the first technology I have seen that could change them by an order of magnitude. If it stays in the racks, the plausible future is a widening split: premium hosted agents running at thousands of tokens per second at premium prices, and local inference keeping its capacity advantage while permanently ceding speed. Local stays the place where things fit; fast becomes something you rent.

I would rather live in the other future, the one with a model card in a PCIe slot. AMD, of all companies, has form here: Strix Halo itself is datacenter-adjacent capability pushed into a consumer box. Whether they do it again with etching is, as far as I can tell, completely open.

The honest caveats

  • The 17,000 t/s figure is a vendor benchmark on an 8B model from 2024 in a quality-degrading 3-bit format. No frontier-class chip exists yet; the HC2 (20B parameters per chip, pipeline parallelism beyond that) is due this summer.
  • KV cache lives in on-chip SRAM, and SRAM is small. The H100 post's lesson applies unchanged: for long-context agentic work the blocker is session memory, not decode speed. How many 100K-token sessions fit in that SRAM fabric is the spec I want before believing the agentic story end to end.
  • The deal has not closed, no roadmap has been announced, and every product claim in this post beyond the HC1's existence is inference or labeled speculation.

The physics is sound, the lock-in objection is weaker than it looks, and the strategic logic is real. What is genuinely undecided is who gets access to the speed. That decision now belongs to AMD, and it will say a lot about what local inference is allowed to become.