OpenAI First-Gen Inference Chip Launch

OpenAI announced its first-generation inference chip, a custom ASIC designed from scratch for large language model inference. The chip was designed and taped out in 16 months starting mid-2024. It is a general-purpose inference chip, tested on models including DeepSeek R1 and Kimi 2.5, and even capable of running the game Doom. The chip is engineered for extreme energy efficiency, with per-watt throughput surpassing Nvidia's Blackwell and even the HBM4-based Rubin. Performance metrics include over 700 token/s/user for DeepSeek R1, over 1400 token/s for GPT OSS, and nearly 700 token/s/user for Kimi 2.5. OpenAI's unreleased Astra AI model participated in the chip's design. The chip team used Codex and GPT Astro to optimize three open-source models to peak performance in two months, with AI-generated modules running 1.5 to 1.8 times faster than human-expert-written ones. Currently an engineering sample, volume production is expected to ramp in 2027, with most capacity deployed by the end of next year. OpenAI's next goal is 100MW deployment scale, starting in its own infrastructure by year-end.
Key insights
AI Competition Shifts to Cost and Supply Chain Integration
The AI field has entered a new phase where competition extends beyond model capability to cost efficiency and full supply chain integration. OpenAI's custom chip represents a strategic move to control its own hardware destiny, directly challenging Nvidia's dominance. The chip's design specifically targets the core bottleneck of modern data centers: power, not budget or floor space.
AI-Assisted Chip Design Accelerates Development
OpenAI's unreleased Astra AI model was deeply involved in designing the chip, which may explain the extremely high energy efficiency and short 16-month development cycle. The chip team used Codex and GPT Astro to optimize three open-source models to peak performance in just two months. AI-generated modules were 1.5 to 1.8 times faster than those written by human experts. This creates an ironic situation where OpenAI models running on Nvidia GPUs are helping design chips aimed at breaking Nvidia's CUDA moat.