Updated todayNew2 sources

Artificial Analysis v5: open-weights gap to closed models widens to 12 points

The gist

The widening gap suggests closed frontier models are pulling ahead of publicly available open-weights alternatives on capability benchmarks. Separately, a commentary argues that independent model evaluation is harder than generally credited, and that identifying the right benchmarks to optimize against is a more difficult problem than improving scores on existing ones.

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

The full picture

Artificial Analysis v5 benchmark results show open-weights models falling further behind frontier closed models. The gap between the open-weights leader and the overall benchmark leader grew from 3 points in v4.3 to 12 points in v5. Kimi K3.1 Pro leads open-weights models with a score of 46, though its weights have not yet been released. Among models with released weights, DeepSeek V4.1 Flash 1017 and GLM-5.3 top the open-weights category at 32. Claude Haiku 5.5 (max) outscores all open-weights models on the benchmark. The gap is most pronounced on HLE-Diamond and Terminal-Bench-Science, where Claude Opus 5.5 (max) scores 68% and GPT-6 Astra scores 63%, compared with Kimi K3.1 Pro at 37% and 34% respectively.

How it developed

28 October 2026

Commentary published arguing benchmark design is harder than benchmark score optimization

Sources

Dean Ball on X