Summary

Load-bearing causal claim: OpenAI's first-gen inference ASIC (Jalapeño, Broadcom, TSMC N3P, HBM4 likely Samsung) is a real, high-volume chip whose A0 lab numbers beat Blackwell on tokens-per-MW and match/beat early Rubin on that metric — without speculative decoding — but it is still engineering samples, 8k1k not AgentX, and production is a 2027 ramp. SemiAnalysis verified InferenceX runs in the lab; they did not run AgentX (the suite that flipped the CUDA-moat read on 08-24). Tape-out Nov 2025 (CoWoS); B0 already in fab (~25% better perf/W; 13.4 PFLOPs MXFP4 @ 700W vs Rubin 17.5 PFLOPs NVFP4 @ 900–1,150W). "The CUDA moat is potentially dead given how fast OpenAI can bring up new models on their silicon" is the author's inference from software bring-up speed, not an AgentX result. First production token "coming soon"; 100 MW is the next hardware goal. Celestica does the rack; AMD EPYC Turin hosts; Broadcom Tomahawk 6 scale-up.

This attaches to existing spines — cuda-moat-erosion-to-nvda-rerate (measured counterweight to yesterday's AgentX "NVIDIA still wins" on a different bench), hbm-cowos-as-binding-bottleneck (HBM4 + CoWoS, Samsung as likely stack), inference-demand-to-wafer-scale-advantage (perf/W as the scoring function because DCs are power-limited). Not a net-new AI-infrastructure chain. AVGO (foundry partner), TSM (N3P + CoWoS), 005930.KS / Samsung HBM, CLS (Celestica) are the named picks-and-shovels. ⚠ Free portion only — NVDA/AMD/Cerebras implications sit behind the paywall.

Article

OpenAI has spent the past couple years quietly building "Jalapeño," an inference chip just announced at Hot Chips. Rumors of a successful tapeout had been swirling for a while. But now we have details. OpenAI invited us to look at their chip, go to their labs to check out how real it is, and benchmark it with our InferenceX suite.

In June, OpenAI unveiled the chip program in partnership with Broadcom, built from a blank slate exclusively for LLM inference. Design work began in the middle of 2024, going from initial team hiring to manufacturing tape-out in ~16 months, an extremely fast ASIC development cycle.

In general first generation chips are not competitive, but OpenAI bucks the trend by being industry leading and beating every Nvidia, AMD, and Google chip we have been able to test on multiple top open source models. OpenAI does this with extreme hardware software codesign. Surprisingly, OpenAI is not over specialization on any specific part of model inference, but instead by focusing on being a general chip that delivers high performance in all scenarios.

A generalized inference chip

Everyone says that OpenAI's chip is specialized for OpenAI models, but that's wrong, OpenAI made a generalized chip for AI inference.

The timelines are insane. It shows that claims that use of AI is being used to accelerate chip design are real.

Jalapeño will ship with HBM4, making this chip one of the relatively early adopters after Nvidia and AMD, even beating the established TPU and Trainium programs. This results in 15.4TB/s of memory bandwidth per package which bests all the other accelerators shipping that are using HBM3E. The 15.4TB/s bandwidth shows its HBM4 can hit 10Gbps pin speeds, which would give it a slight edge over the 9.6Gbps Nvidia is getting out of its HBM4 in Rubin. The HBM is likely provided by Samsung.

The following is our headline perf/W result, looking at token throughput per All-in utility MW. Jalapeño smokes every other chip. All this is done without Multi Token Prediction (MTP), while the other chips on the chart are the best performing configs of each respective SKU, all with MTP.

Jalapeño beats Blackwell on perf/W across almost all scenarios without being tuned for any specific point in the curve. At low concurrency scenarios, Jalapeño demonstrates remarkable interactivity, hitting over 700 tokens per sec per user at concurrency 1 on the DeepSeek R1 model.

Incredibly, this is all achieved with single-token prediction (STP), no speculative decoding and no prefill-decode disaggregation. In addition to DeepSeek R1, we also got to see some other models, including Kimi-K2.5 and GPT-OSS which ran at approximately 1,400 tok/sec/user. For all models, we confirmed that Jalapeño's GSM8k evals attained results on par with Nvidia chips.

Some caveats on this. First, all numbers are provided to us by OpenAI. We verified the InferenceX runs in person in the lab, but we did not run the full suite of InferenceX benchmarks nor have we seen AgentX results. AgentX is our preferred suite for comparing chip performance due to the datasets' long context and multi-turn characteristics that reflect the cache behavior of realistic production workflows. Frameworks that perform well on 8k1k may perform worse on AgentX as real production loads stress components like routers, prefix cache mechanisms, cache management, offload infrastructure, etc.

Second, we believe that comparison to Blackwell is somewhat incomplete and unfair. Jalapeño is really competing against chips like Rubin that also use HBM4. Vera Rubin systems are starting to ship to customers right now, while it will still be some time before OpenAI has anything beyond engineering samples of Jalapeño.

Vera Rubin NVL72 delivers 5.4x the perf/MW of GB200 NVL72 as we described in our article analyzing the NVIDIA performance claims in their launch with CoreWeave last month.

Third, the models being tested are not on the open frontier. NVIDIA and AMD have published results on larger models such as DeepSeek V4 Pro and Kimi K3, using AgentX.

Performance Analysis

OpenAI designs for perf/W. The reason is simple: OpenAI is currently limited by datacenter power, not by budget or floorspace, and thus tokens per MW is paramount. At Computex 2026, Jensen said that perf/W, reliability and long lifetime are the core features of future GPUs. To quote: "If you have 1 gigawatt of power, then throughput per watt is revenue".

This was emphasized by Nvidia during the Vera talk at Hot Chips 2026 while showing the same revenue graph: "The data center is power limited today."

On this front, even when compared with Rubin, Jalapeño wins. OpenAI's Jalapeño has STP output token throughput per MW surpassing Vera Rubin's MTP results that NVIDIA and CoreWeave published in July.

On perf/TCO, Vera Rubin and Jalapeño are head-to-head, producing almost the same number of output tokens per $. However, as previously mentioned, Jalapeño's results are obtained without speculative decoding and Vera Rubin's results use speculative decoding. Speculative decoding leads to a ~3-5x reduction in cost per token.

Architecturally, OpenAI chose not to disaggregate prefill and decode (PD) across separate chip pools. The draft model and main model share the same chips and fabric.

On Kimi K2.5 (which Cursor Composer 2.5 is based on), Jalapeño reaches nearly 700tok/s/user and more than 9x the next best performing chip at 100tok/s/user.

On GPT-OSS, it's another bloodbath. Jalapeño's iso-interactivity throughput per MW is nearly double GB200's highest throughput point and more than 50x GB200's concurrency 1 point.

These results are impressive! However, we have to nitpick: they're just 8k1k, a much easier workload to tune for, and there are no AgentX runs yet.

Specs

All these results were gathered on the A0 stepping of Jalapeño, just 9 months into the program. But there is already a B0 stepping that is currently in the fab! B0 has optimizations that deliver roughly a 25% perf-per-watt improvement over the earlier A0 silicon. Specifically, the B0 stepping delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized compute die that is manufactured on TSMC's N3P. This compares to 17.5 PFLOPs of dense Rubin NVFP4 for a single Rubin compute die that is similar size and on the same node.

This is more respectable considering Jalapeño's TDP is only 700W compared to Rubin's at 900-1,150W per compute die.

OpenAI taped out Jalapeño in November 2025, or more specifically, this was a tape out of the CoWoS design, not just the top die silicon. Within 9 months of that Nov 2025 tapeout, and with only 3 months of bring-up on actual silicon, OpenAI has already delivered very good results with Jalapeño. This is all the more impressive as the team is starting from zero on the software stack.

Meanwhile, Rubin's CoWoS tape out was completed in October 2025, a month earlier, and yet the only early results we have seen are from CoreWeave's engineering samples. Nvidia has not let us test and release benchmarks in the same way that OpenAI has, indicating their chip software is still immature. The CUDA moat is potentially dead given how fast OpenAI can bring up new models on their silicon.

While OpenAI has engineering samples of Jalapeño, production is currently scheduled to gradually ramp over 2027 with most of the output currently scheduled for the end of next year.

Suffice to say, OpenAI Jalapeno is a real high volume ASIC.

Software

OpenAI writes Jalapeño kernels like assembly. Each kernel gets hand-tuned code, some running to ~3,000 lines, backed by correctness checks and a custom sanitizer. Early kernel work was human-in-the-loop rather than fully automated, but this shifted with a more scaled-up, internal version of Codex. The internal serving engine is called "Teacup". Interestingly, OpenAI had no internal implementation of MLA kernels until they benchmarked DeepSeek with InferenceX.

In a weird twist of fate, OpenAI models like GPT 5.6 Sol, which currently run on NVIDIA GPUs, have been used to design a chip that poses a real threat to the CUDA moat - NVIDIA's own GPUs are helping usher in their potential successor in real time.

Rack (Celestica / AMD / Broadcom)

The Jalapeño System at the rack unit level consists of a CPU host rack and an ASIC rack. The host rack houses 16 host CPU trays named "Katsu," each corresponding to one of the 16 ASIC trays, named "Vindaloo." Each host houses two Turin-class AMD EPYC CPUs with 1.5TB of DRAM. The system level design is done in partnership with Celestica.

The ASIC rack consists of 16 Vindaloo trays and 8 scale up switch trays named "Chana." Each Vindaloo tray consists of 8 Jalapeño ASICs, making up a total of 128 Jalapeño ASICs per rack. The ASICs are connected to each of the Chana switch trays via a copper cable backplane, just like that of Nvidia's Oberon.

Power provisioning to a sidecar host rack draws roughly 50kW provisioned (31kW in production), and the ASIC rack draws 130kW, making the total two rack system roughly 160kW. That's basically a double-wide GB300 rack in terms of power draw.

OpenAI can connect up to 2,048 Jalapeño XPUs within a single scale-up network. Six Chana switches in the middle are for the local domain, which come with one 102.4T Tomahawk 6 switch ASIC each.

On deployment, OpenAI is partnering with neoclouds and is gathering reliability data with datacenter partners through January while optimizing dock-to-rack rollout time.

What's Next (paywall)

Next, we talk about the future of Jalapeño, whose first production token is coming soon. The next goal is 100MW, and the hurdles will mostly be hardware: How much can they produce, how well can they deploy and operate datacenters, how do they handle monitoring, and resiliency, etc. The software is already proven, and with internal models, every software headstart is easily caught up to. Behind the paywall we will discuss implications for NVIDIA, AMD, Cerebras, and other chip companies who have signed deals with OpenAI in the coming years.

[Clipping is the free RSS portion. Paywalled implications for NVDA / AMD / CBRS not fetched. Full post: https://newsletter.semianalysis.com/p/openai-jalapeno-better-than-nvidia]