Preference cascade broadens across labs; UN High Commissioner Turk declares AI existential risk; Congress demands action; Pachocki voices concern; Schulman proposes OpenAI-Anthropic pacing collaboration; CBS News interview airs
Anthropic Researcher's Exit Sparks AI Extinction Preference Cascade and UN Alarm
Active researchers at multiple frontier AI labs are publicly stating beliefs in meaningful extinction probability, moving what had largely been an internal conversation into open public and political discourse at scale. Government attention at both the congressional and international level is now directly engaged with the question.
The full picture
A public preference cascade has spread Jacob Coxon's concerns beyond his individual resignation, with researchers across Anthropic, OpenAI, and Google DeepMind openly stating beliefs that AI could cause human extinction, and the UN High Commissioner for Human Rights formally declaring advanced AI an existential risk. Coxon, who previously worked on pretraining at OpenAI before joining Anthropic in May 2026, resigned four months into his tenure, two months before his equity would vest, forfeiting his entire stake. His public post warning about AI extinction risk reached over 160 million views. His core claim centers on recursive self-improvement, which he says insiders believe is roughly one year away and refer to as crunch time and endgame, and which he said could make AI systems uncontrollable by the end of 2027. On NBC News he warned that many executives and senior researchers genuinely believe there is a 'substantial probability' AI could kill everyone, and he told The Wall Street Journal that Anthropic is 'far and away the most responsible player in the space' while adding that neither company is truly safe. Anthropic's Alignment Science lead Evan Hubinger publicly stated he personally estimates greater than 10% probability of AI killing all humans within the next decade, and that Anthropic has no clear plan to solve alignment for superintelligence. OpenAI Chief Scientist Jakub Pachocki stated he is concerned 'no one is prepared for the consequences of a continued rapid rise in machine intelligence.' Google DeepMind researchers also spoke out, despite what a former GDM communications employee described as a prohibition on external extinction risk communication that was in place at GDM in 2018. UN High Commissioner for Human Rights Volker Turk called for international red lines and announced plans to write to AI companies urging immediate safety measures. Dozens of members of Congress demanded urgent action. John Schulman separately called on OpenAI and Anthropic to jointly develop a pacing proposal, while the Senate committee responsible for commerce and technology has no AI hearings scheduled for the next four months. A public backlash has emerged that critics say conflates distinct AI problems, including data center energy use, cybersecurity, and job displacement, in ways that could lead to poorly targeted regulation.
How it developed
Axios reports equity forfeiture; NBC News interview airs; Interconnects newsletter analyzes the viral spread and argues extinction framing is overblown
Wall Street Journal reports Coxon's resignation; Ars Technica covers Hubinger publicly backing the warning with greater than 10% extinction estimate
Sources
- President Trump dismissed AI extinction concerns, prioritizing competition with China as frontier-lab researchers urged …
- Axios: Jacob Coxon quit over safety concerns 2 months before any of his equity vested.
- WSJ just reported: Anthropic researcher Jacob Coxon is quitting AI because he thinks self-improving models could become …
- 🟡 Algorithmic anxiety
2 more sources
Want this in your inbox?
I send one email each morning with the stories that moved. If you would rather just read here, that works too.
Subscribe free