The Information Machine
Concluded·New·first covered 29 Aug 2026·2 sources·updated 29 Aug 2026

Open ASR Leaderboard Hindi and Indian English benchmarks

The gist

Open ASR Leaderboard adds Hindi and Indian English benchmarks

Benchmark coverage shapes which models get built and improved; languages absent from major evaluation sets leave performance gaps invisible to developers. Adding Hindi and Indian English to the Open ASR Leaderboard creates a public accountability mechanism for model quality on those languages.

The full picture

The Open ASR Leaderboard, operated by HuggingFace, has added Hindi (Monsoon hi) and Indian English (Monsoon en-IN) benchmarks through a partnership with VoiceArena. These are the first Global South languages included on the leaderboard. The benchmarks cover 4,888 speakers across four speaker-disjoint splits, with audio recorded on speakers' own phones during spontaneous conversation rather than in studio conditions. Each language has a public split for self-scoring and a private held-out split for more robust evaluation. The evaluation uses OIWER against reference lattices to handle cases where a single written transcription is insufficient.

How it developed
29 August 2026

HuggingFace's Open ASR Leaderboard added Hindi and Indian English benchmarks on August 28 through a partnership with VoiceArena, making these the first Global South languages on the leaderboard.

The two benchmarks, named Monsoon hi and Monsoon en-IN, cover 4,888 speakers across four speaker-disjoint splits, with audio recorded on speakers' own phones during spontaneous conversation rather than in studio conditions. Each language includes a public split for self-scoring and a private held-out split, with OIWER used against reference lattices to handle variation in written transcription.

Sources
The daily email

Want this in your inbox?

I send one email each morning with the stories that moved. If you would rather just read here, that works too.

Subscribe free