A company that bakes AI models directly into silicon wafers has been acquired by chip giant AMD. Founded in Toronto in 2023, startup Taalas claims its custom silicon runs AI models 48 times faster than top-tier GPUs, outputting up to 17,000 tokens per second while consuming far less power. Financial terms were not disclosed, and the transaction is expected to close in the fourth quarter of this year. As the AI industry fights a price war over the sheer cost of running models, AMD’s acquisition is a heavy bet on a radically different technological path.
What Is a Model: A Long String of Numbers, a Recipe
To understand “etching AI into silicon,” one must first grasp what an AI model actually is. A trained large language model is essentially a massive sequence of numbers—ranging from billions to trillions—known in the industry as “weights.” These numbers determine how the AI formulates its output: seeing “How is the weather,” it predicts “today.” To use an analogy, a model is like a recipe—exact grams of flour, sugar, and baking minutes—all represented as numbers describing every step of the process.
How does a GPU run AI? It acts as universal kitchenware. The recipe sits on the counter, ingredients are laid out, and the GPU reads the recipe while measuring and cooking simultaneously. The advantage is versatility: it can cook any recipe imaginable. The drawback is inefficiency: every single dish requires re-reading the recipe and re-measuring ingredients, wasting time and electricity.
Taalas took a different route: etching the recipe directly into the bottom of the pan. The pan itself inherently knows how to make that specific dish—no recipe reading, no ingredient measuring required; just turn on the heat, and it’s done. This “pan” is customized for a single dish—a specific AI model. The moment the chip leaves the fabrication plant, the model’s weights are permanently hardwired into the silicon as physical circuits. This chip can only run that single model and nothing else, but when running it, the speed is astonishing.
Figure: Taalas’s HC1 demonstrator board, where the model is permanently etched into silicon as hardware circuits. Source: taalas.com
17,000 Tokens per Second: A Whole Book Written in Seconds
Taalas isn’t just making theoretical promises. In February of this year, it taped out its first test chip, the HC1, built on TSMC’s 6nm process. Etched onto it was Meta’s open-source model Llama 3.1 8B—an 8-billion-parameter model. In real-world tests, it generated 17,000 tokens per second. A token is the fundamental unit of text for AI, representing roughly half to three-quarters of a word.
To put this speed in perspective for everyday users: services like ChatGPT currently stream text to users at a rate of dozens of tokens per second, barely keeping pace with human reading speed. 17,000 tokens per second corresponds to tens of thousands of words per second—a 100,000-word book generated in roughly ten seconds. Official comparison benchmarks cite a speedup 48 times faster than NVIDIA GPUs. Using NVIDIA’s H200 GPU as a baseline (~230 tokens/sec versus 17,000 tokens/sec), the difference approaches 74 times. Regardless of baseline metrics, the conclusion remains the same: a speedup of several orders of magnitude.
Figure: Taalas’s official chart comparing single-user tokens per second for Llama 3.1 8B, with NVIDIA H200 as baseline. Source: taalas.com
From an engineering perspective, these figures represent ideal single-user, single-model benchmark data. Real-world performance in data centers will see some discount. But even cut in half, it is more than enough to rewrite the economic math of AI inference.
Why This Matters: Every AI Answer Burns Money
AI has two main cost phases. Training—feeding data to build the model—costs upwards of a hundred million dollars per run, but only happens once. Inference—calculating responses every time a user prompts the live model—is an ongoing, continuous operational expense. Every time you query ChatGPT, GPUs behind the scenes are running at full load, consuming electricity and capital.
Inference cost is currently the single largest bottleneck to widespread AI adoption. Over the past two years, tech companies have engaged in an inference price war, continuously slashing API prices by sacrificing margins for market share. Taalas offers an architectural exit: once the model is etched into silicon, there is no need to repeatedly read and transfer hundreds of gigabytes of weights across memory buses—the circuitry itself is the data. In a February interview, Taalas claimed that the cost of etching a model into silicon is 100 times cheaper than training a frontier model. While this company metric awaits independent verification, the underlying direction is credible: no matter how expensive a mask set or tape-out is, it pales in comparison to training a trillion-parameter model.
Universal Kitchenware vs. Custom Pot: The Clash of Two Paradigms
This brings the fundamental architectural divide into focus.
GPUs represent the general-purpose path: they can run any model. When models evolve, software updates suffice; the hardware remains untouched. The compromise is efficiency—to calculate “anything,” the silicon is packed with broad, generic logic that consumes power, generates heat, and limits throughput.
Taalas represents the application-specific path: designed for one model alone, with ultra-simplified circuitry that is fast, power-efficient, and cheap to run. The trade-off is rigidity—the moment the chip leaves the fab, the model is frozen in time. Physical game cartridges operate on this exact logic: plug it in and play instantly with blazing load times, but if a new game comes out, you must buy a new cartridge. Digital downloads can be updated anytime, but require waiting, bandwidth, and processing overhead.
AI models currently iterate on a monthly basis. The model etched inside Taalas’s HC1 chip dates back to mid-2024, which is already dated by today’s standards. When a model updates, the hardwired circuits cannot morph with it. Minor tweaks can be patched on-the-fly (via LoRA adapters, akin to sticking sticky notes on a printed recipe book); major architectural shifts require a full re-tapeout—manufacturing brand-new silicon. Taalas claims that switching models only requires redesigning two metal layers, making it far cheaper than designing from scratch, yet it still demands factory capacity and production lead times. Which model is worthy of being permanently frozen into silicon? Only those that have been massively validated and destined for long-term service. Bet on the wrong model, and racks of chips become e-waste.
Why AMD Is Making This Bet
NVIDIA commands the lion’s share of the AI chip market. AMD has chased for years, closing the hardware performance gap, yet software ecosystem friction remains a hurdle—AI developers worldwide are deeply accustomed to NVIDIA’s CUDA toolchain. Fighting head-on in software ecosystem dominance is tough, so AMD is picking a new battleground: wagering on model-specific inference silicon.
Competitors aren’t standing still either. In December of last year, NVIDIA signed a $20 billion licensing deal with inference chip maker Groq, targeting the exact same objective: making AI inference faster and cheaper. Both tech giants are betting on the same strategic vector from different tactical angles.
Taalas founder Ljubisa Bajic was previously CEO of Tenstorrent and an AMD alumnus—making this acquisition a homecoming of sorts. The team will join AMD’s AI group. According to analysis by The Register, AMD’s intent may be pairing Taalas chips with its own Instinct GPUs: GPUs handle the initial prompt processing and context setup, while Taalas chips take over for lightning-fast token generation, leveraging each architecture’s core strength. AMD’s official statement remained understated, noting the acquisition aims to “provide the right compute solution for every AI workload.”
Figure: Product demonstration screenshot from Taalas’s official website. Source: taalas.com
What This Means for Everyday Users
The operational cost of AI ultimately determines the price of consumer AI services. If hardwiring models into silicon can slash inference costs by an order of magnitude, AI will become vastly cheaper and faster, enabling deployment inside phones, TVs, automobiles, and home appliances—places that cannot house racks of power-hungry GPUs.
On Hacker News, commentators mused: Apple should buy this company and etch AI models directly into iPhone chips for instant, zero-battery-drain responses. While a natural impulse, reality is still distant: the HC1 test board is sizeable on its own, and it etches a model from two years prior. Having an outdated model permanently hardwired inside your phone creates an immediate dilemma when a superior model releases—buy a new phone?
This architectural debate has no quick resolution. General-purpose GPUs bet that “models will keep evolving endlessly,” while custom silicon bets that “a few core models will endure long enough to warrant dedicated hardware.” The former prioritizes flexibility; the latter pursues extreme efficiency. Winner-takes-all will be decided by the race between model iteration cadence and silicon fabrication economics. With the deal closing in Q4, AMD is putting real capital behind its answer. We will be watching closely.
Reference Links:
- The Register: AMD acquires AI chip startup Taalas to boost inference performance by etching models into silicon
- Hacker News: AMD acquires Taalas Discussion Thread (item?id=49201970)
- AMD Official Press Release: AMD Acquires Taalas to Advance Compute Solutions for Rapidly Growing AI Inference Market
- CNBC: AMD buys Taalas, startup that hardwires AI models into its silicon
- EE Times: AI Chip Startup Taalas Acquired By AMD
- Taalas Official Website: HC1 Technology Demonstrator Product Page
- Medium: A Look at Taalas’ HC1 Chip Reaching 16,000 Tokens per Second
