OpenAI has introduced its first specialized chip, Jalapeño, for inference tasks of large language models. Developed jointly with Broadcom, the accelerator is focused on processing requests to models like GPT and Codex, not on training. The announcement at the Hot Chips conference in August 2026 was accompanied by initial benchmarks showing a notable advantage in tokens per watt and reduced latencies.
A key technical aspect is the architecture optimized for characteristic LLM inference patterns: high throughput with low latency and efficient memory usage. Unlike general-purpose GPUs, Jalapeño focuses on specific operations such as attention and feed-forward in transformers. Early tests on SemiAnalysis InferenceX demonstrate 1,5–1,9 times more work per watt compared to Nvidia Blackwell, and in certain scenarios up to 104-fold increase in tokens per watt is claimed—a figure that requires independent verification.
The testing methodology includes comparisons with systems based on Nvidia GB200/GB300, normalized by the chip's stated power (700 W, with real consumption up to 550 W). Results cover both OpenAI's internal models and external ones—DeepSeek R1 and Kimi K2.5. This strengthens the argument for cross-model applicability, but the lack of detailed ablations and full dataset descriptions leaves questions about reproducibility.
In the AI chip landscape, Jalapeño occupies a niche of specialized ASICs for inference, similar to Google's TPU or Amazon's Trainium/Inferentia. Unlike the approach of Meta or Microsoft, OpenAI is betting on tight integration with its own models and a multi-generational platform. This brings the strategy closer to Google, but accelerates the development cycle to nine months thanks to the use of AI in design.
For the industry, this means a potential reduction in inference costs in data centers, which is especially important when scaling agentic systems and APIs. Lower latency (1,7–3,6 times) allows serving more users without compromising between performance and response speed.
It remains unclear how sustainable the metrics will be under real load in 2027, when competitors release new generations. Independent tests and disclosure of architecture details will be key to assessing long-term impact.
The development of Jalapeño underscores that inference efficiency is now determined not only by model size, but also by hardware specialization for specific workloads.
