OpenAI has been building its own chip for a while now. There are finally some numbers — and they’re serious.
Jalapeño is OpenAI’s first custom inference chip, meaning the hardware that runs already-trained models in production. The idea: build a chip from the ground up for exactly that job, instead of repurposing GPUs that were really designed for training.
Testing ran on InferenceX, a public benchmark from SemiAnalysis, across three open models: GPT-OSS 120B, DeepSeek R1 at 670 billion parameters, and Kimi K2.5 at a trillion. Across all three, Jalapeño delivered 1.5 to 1.9 times more compute per watt at peak throughput and 1.7 to 3.6 times lower latency than the comparison systems. On highly interactive workloads — exactly where agents run many steps back to back — it hit up to 4.1 times the performance.
The interesting part is the combination. Normally you have to choose: high throughput or low latency. Jalapeño manages both in one architecture, because OpenAI designed the chip, memory, network, and software together around real language-model workloads. For context: OpenAI rates the chip at 700 watts, and it stayed under 550 in these tests — compared against Nvidia’s GB200 and GB300, which are rated at 1,200 and 1,400 watts.
What sticks with me most is a different point. OpenAI used AI to design the chip — from first draft to finished tapeout in nine months. And for selected building blocks, the AI-written implementations ran 1.5 to 1.8 times faster than the ones from human experts. Models that weren’t even in the original plan were brought up within two months using Codex and a model called GPT-Astra.
There’s a loop closing here: AI helps build the hardware that AI then runs on. The better the models get, the faster the next chip generation can be optimized.
OpenAI plans to start deploying Jalapeño in its own infrastructure by the end of the year. It’s the first of several planned generations, with Gen 2 already in development. Nvidia stays in the picture, though — OpenAI keeps buying for training and inference wherever compute comes from.
Honestly? For end users, nothing changes right away. But when inference gets cheaper and faster, you feel it eventually — in more responsive agents, lower prices, more capacity. And the race for the chips underneath is at least as interesting right now as the race for the models on top.
Source: OpenAI: Jalapeño’s first results