David Sacks calls for Anthropic IPO to be paused pending investigation of researcher's claims
Anthropic Alignment Lead Puts >10% Odds AI Kills All Humans This Decade
A sitting alignment lead at a major AI lab publicly assigned greater than 10% probability to human extinction from the technology his company builds, while acknowledging his company has no plan to address the underlying alignment problem. The disclosures coincide with Anthropic's IPO marketing.
The full picture
Anthropic Alignment Science lead Evan Hubinger publicly stated he personally estimates greater than 10% probability that AI kills all humans within the next decade, while acknowledging Anthropic has no current plan to solve alignment for superintelligence and is not clearly on track to develop one. The statement backed Jacob Coxon, a safety researcher who resigned from Anthropic after roughly four months, forfeiting his entire equity stake because Anthropic's vesting begins at six months. Coxon, who moved to Anthropic from OpenAI for its safety focus, warned that frontier AI companies are 'gambling with our lives' and that recursive self-improvement could leave AI systems out of control by the end of 2027. Hubinger also said recursive self-improvement is happening faster than Anthropic previously thought, and that senior AI researchers privately express more fear about AI risk than they do publicly. The statements coincide with Anthropic's IPO marketing phase; David Sacks publicly called for the IPO to be paused pending investigation of the researcher's claims.
How it developed
Hubinger publicly backs Coxon, stating greater than 10% extinction estimate and no plan to solve alignment for superintelligence
Sources
- Axios: Jacob Coxon quit over safety concerns 2 months before any of his equity vested.
- Anthropic is approaching IPO marketing while a researcher at the company publicly forecasts a greater than 10% chance of…
- The Information just published this piece
- WSJ just reported: Anthropic researcher Jacob Coxon is quitting AI because he thinks self-improving models could become …
Want this in your inbox?
I send one email each morning with the stories that moved. If you would rather just read here, that works too.
Subscribe free