Baseline · Inference Silicon Tracker v1.0

Welcome to the Inference Silicon Tracker

This is the first weekly release of the Inference Silicon Tracker — a companion to the Inference Engines Tracker and Frontier Models Tracker, covering the hardware layer: the chips, architectures, and companies competing to power AI inference after the GPU era.

What’s tracked at baseline (June 2026)

13 chips and architectures across 6 families:

GPUs (incumbents)

• NVIDIA H100 SXM5 — the workhorse. 989 TFLOPS FP16 sparse, 3.35 TB/s HBM3, 80 GB. Still the default for production inference.
• NVIDIA B200 / GB200 — Blackwell generation. 4.5 PFLOPS FP4, 8.0 TB/s HBM3e, 192 GB. NVL72 rack systems (72 GPUs, 13.8 TB total) shipping to hyperscalers.
• AMD MI325X — 288 GB HBM3e makes it the memory-capacity leader among GPUs. ROCm 6.x ecosystem maturing.

Wafer-Scale

• Cerebras WSE-3 — 46,225 mm² full-wafer chip. 44 GB on-chip SRAM, 21 PB/s bandwidth. 6.7× faster than best GPU cloud on a 1T-parameter model. GA.

Dataflow / LPU

• Groq LPU (Groq 3) — deterministic compile-time scheduled inference. Sub-millisecond first-token latency. Acquired by NVIDIA for $20B in December 2025. Groq 3 debuted at GTC 2026.
• SambaNova SN50 RDU — 5th-gen Reconfigurable Dataflow Unit. 5× speed claim, 3-tier memory architecture for agentic workloads. Shipping H2 2026.

Inference ASICs

• Etched Sohu — transformer hardwired into silicon. 20× throughput claim vs GPU. TSMC 3nm. Not publicly available.
• Fractile (unnamed chip) — SRAM memory-compute fusion, no DRAM/HBM. 100× speed, 90% cost reduction claimed. $220M Series B, May 2026. Expected 2027.
• d-Matrix Corsair — digital in-memory compute. 128 GB DRAM with co-located compute. Customer sampling.

RISC-V

• Tenstorrent Blackhole — RISC-V Tensix tile mesh, open-source TT-Metal SDK, open IP licensing. Led by Jim Keller. Sampling.

Cloud-Only ASIC

• Google Trillium (TPU v6) — 4.7× ML compute vs TPU v4. GA on Google Cloud.
• AWS Trainium2 / Inferentia3 — 3.7× throughput vs Trainium1. GA on AWS.

---

Key moves since Jan 2026

Fractile $220M Series B (May 2026): The UK startup, founded in 2022 by Oxford PhD Walter Goodwin, closed a $220M Series B at ~$1B valuation. Backers include Accel, Peter Thiel’s Founders Fund, Felicis, and 8VC. Anthropic reported in early talks for chip supply. The architecture — SRAM memory-compute fusion with no off-chip DRAM — is the most radical memory-wall attack yet. Hardware expected 2027.

NVIDIA acquires Groq ($20B, Dec 2025): The largest AI chip startup acquisition on record. Groq’s deterministic LPU architecture and GroqCloud’s developer ecosystem (1M+ registered) are now inside NVIDIA. Groq 3 LPU was unveiled at GTC 2026.

SambaNova SN50 + Intel deal (Feb/Apr 2026): SambaNova’s 5th-gen RDU launched alongside a $350M+ funding round and a novel split-inference architecture partnership with Intel — pairing Intel Gaudi 3 CPUs with SN50 accelerators for combined deployment.

Cerebras IPO: Cerebras (CBRS) completed its IPO in late 2024. The company’s Condor Galaxy 3 cluster (9× CS-3 systems) deployed with Abu Dhabi’s G42 demonstrated 21,000 tokens/sec on Llama 3 405B.

---

What to watch

• Fractile’s first silicon benchmarks — if the memory-compute fusion architecture delivers even 30% of its claimed 100× improvement, it changes the economics of frontier inference.
• Groq 3 integration into NVIDIA NIM — will the LPU’s latency advantage survive assimilation?
• Etched Sohu availability — still no public GA date as of June 2026. Watch for Hot Chips 2026 papers.
• AMD MI350X — TSMC 3nm MI325X successor, expected late 2026. If ROCm ecosystem matures further, AMD becomes a credible NVIDIA alternative.
• Tenstorrent Galaxy server system — 32 Blackhole cards; could be the first truly open server-grade inference stack.