2026 07 23 Feed Semianalysis Vera Rubin Nvl72 VS Gb200 Inference

Summary

SemiAnalysis argues Vera Rubin NVL72's inference-economics advantage over GB200/GB300 is driven primarily by relieving the memory bottleneck — the exact link at the center of hbm-cowos-as-binding-bottleneck and hbm-supply-bottleneck. Rubin gets 2.8x higher global memory bandwidth via 3D-stacked HBM4 and 288GB per package (vs GB200's 192GB), so larger models fit in fewer packages. The chain to suppliers: higher-density HBM4 adoption = volume opportunity for SK Hynix / Samsung (micron is the third HBM vendor by extension), and Rubin's "simpler cableless compute tray" reduces CoWoS/packaging manufacturing complexity vs Blackwell's problematic copper backplane, potentially improving yields/lead times (tsmc CoWoS). Corroborates the HBM/CoWoS-as-binding-bottleneck thesis from an independent written source (diversifies away from the podcast/earnings legs).

Paywall note (partial: true). SemiAnalysis is paid-subscriber-gated; the article terminates at "This post is for paid subscribers." Captured text below is the free preview only — the paywalled section likely held the detailed supply-chain vendor analysis and ticker specifics. Free portion only; not fabricated.

Article (free preview — verbatim claims as returned)

Central thesis. Vera Rubin NVL72 delivers substantially better inference economics than GB200/GB300 — claimed ~5.4x performance-per-megawatt and ~5x performance-per-dollar over GB200 NVL72 — but the gains narrow significantly when compared against the contemporary GB300 NVL72 baseline rather than GB200's early-2025 software stack.

Performance claims. Per-MW gains: ~2x at low interactivity (100 tok/s/user), peaking ~4x at 200 tok/s/user, then 5.4x at 300 tok/s/user (where GB200 cannot operate at all). Per-TCO: ~1.5x cheaper at low speeds, improving to 3–5x in mid-to-high interactivity bands.

Primary bottleneck relieved — memory bandwidth & capacity. Three constraints:

1. HBM bandwidth — 2.8x higher global memory bandwidth via 3D-stacked HBM4 vs Blackwell Ultra.
2. HBM capacity — 288GB per package vs GB200's 192GB, enabling larger models (e.g., Kimi K3's 2.8T params fit in ~4 Rubin packages vs 6 in NVFP4-based Blackwell).
3. CPU DRAM — higher capacity for long-context serving. "Weight bandwidth limits decode at low batch size," making Rubin's LUT-based tensor core (3.125 bits-per-weight vs 4.5 for MXFP4) impactful for decode throughput.

Supply-chain implications (free portion). HBM vendors (SK Hynix, Samsung): volume opportunity as Rubin's 2.8x bandwidth drives higher-density 3D-stacked HBM4 adoption. TSMC/CoWoS: Rubin's simpler cableless compute-tray design suggests reduced manufacturing complexity vs Blackwell's copper backplane, potentially improving yields and lead times. (No explicit ticker list in the free content.)

Caveats the authors flag on the headline 10x figure. Compares Rubin (early-stage) against GB200's 2025 software baseline; uses DeepSeek R1 671B ("not widely used anymore"); single-turn 8k/1k context doesn't stress Rubin's 288GB advantage (long-context agentic workloads would show larger gains); pre-production Dell ES rack without scale-out fabric testing.

Secondary innovation — LUT-based tensor core. Rubin introduces a 3-bit programmable lookup-table weight decompression inside the Tensor Core — "first NVIDIA Tensor Core that reconstructs a non-uniform codebook inside the MMA" — enabling asymmetric quantization vs Blackwell's uniform block scaling.