Miles Brundage posted on AI risk probability; Kevin Roose acknowledged his capability-virtue assumption was wrong; Volker Turk called for international AI safety red lines
Pachocki, UN Official, Journalist Flag AI Alignment Gaps Publicly
Pachocki's essay, written from the position of OpenAI's chief scientist, directly challenges the assumption that current safety mechanisms scale with model capability. Turk's call for international red lines and Roose's public reversal on capability-virtue correlation reflect similar concerns reaching official and media circles.
The full picture
OpenAI Chief Scientist Jakub Pachocki published an essay on September 6, 2026 titled 'An Alien Mind,' arguing that no AI lab has solved alignment and that chain-of-thought monitoring, a primary safety mechanism, is becoming less reliable as models grow more capable. Pachocki also distinguishes between goal alignment and value alignment as separate unsolved problems, and argues AI is 'grown more than designed,' the product of optimization rather than deliberate construction, meaning its behavior cannot be assumed to follow human principles. He said future AI progress will be increasingly bottlenecked by confidence in monitoring rather than by compute or data. Separately, UN High Commissioner for Human Rights Volker Turk declared that advanced AI poses an existential risk to humanity and announced plans to write to AI companies urging immediate risk-reduction steps, calling for international red lines among countries hosting AI and involved in its supply chains. New York Times journalist Kevin Roose publicly acknowledged that his longstanding belief that more capable AI models would naturally exhibit better values was mistaken, recognizing that intelligence and virtue are independent properties. AI safety researcher Miles Brundage stated that while he does not personally emphasize probability-of-doom framing, many informed people in the AI industry believe the real risk of catastrophic AI outcomes exceeds 10%, and that even 1% would be unacceptably high.
How it developed
Coverage of Pachocki's essay in multiple outlets, including analysis of goal vs. value alignment distinction
Sources
Want this in your inbox?
I send one email each morning with the stories that moved. If you would rather just read here, that works too.
Subscribe free