OpenAI Releases MentalHealthBench, Built With 80+ Clinicians

OpenAI has released MentalHealthBench, an open benchmark for evaluating AI model performance in realistic mental health conversations. The benchmark was built with more than 80 licensed psychologists and psychiatrists from 22 countries, covering 19 languages and nearly 20 subspecialties. Each synthetic conversation uses a custom expert rubric, with criteria requiring agreement from at least 2 of 3 reviewers and no contradiction from the third. GPT-6 Astra scores 57.3 on the benchmark compared to GPT-4o's 32.1. GPT-5.6 Sol serves as the LLM judge to grade model answers against human-written criteria. OpenAI is releasing the benchmark openly so other researchers can examine the methods, run their own evaluations, and build on the work. Prior mental health AI evaluations focused on emergencies and broad safety, leaving everyday and ambiguous conversations under-measured.