OpenAI's Jalapeño inference chip

A second Semianalysis report published August 30 put Jalapeño's efficiency at approximately 11 million tokens per second per megawatt at around 100 tokens per second per user, roughly twice the best Blackwell-class GPU configurations at comparable interactivity.

The report also disclosed that Jalapeño's published benchmarks use single-token prediction without speculative decoding, while competing chips in those comparisons run with multi-token prediction enabled in their best-performing configurations.

Coverage on August 27 of OpenAI's August 25 Jalapeño benchmarks specified performance gains for highly interactive workloads at 2.1 to 4.1 times, giving a lower bound absent from earlier reporting.

The same coverage introduced a safety argument: an OpenAI executive identified as roon contended that inference at this speed creates threat models where misaligned frontier models could infiltrate systems faster than human responders can act, requiring autonomous detection and shutdown rather than monitoring alone.

Coverage following the August 25 benchmark release confirmed Jalapeño was built in partnership with Broadcom and fabricated by TSMC.

The chip handles inference only, leaving OpenAI still dependent on Nvidia for training workloads; a small batch of Jalapeño-powered systems is planned for 2026, more in 2027, and two additional chip generations are in development.

SemiAnalysis published an analysis August 26 extending OpenAI's Jalapeño chip benchmarks, released August 25, to Nvidia's upcoming Vera Rubin generation, finding Jalapeño beats Vera Rubin's published output-token throughput per megawatt.

SemiAnalysis identified electricity as OpenAI's primary design constraint, described a networking architecture making thousands of chips function as a single inference machine, and suggested OpenAI may be designing for models with tens of trillions of parameters or million-token context windows.

Supply chain checks show Nvidia holding over 80% of AI server market, with ASIC-based servers at 8-11% of shipments