Paul Christiano resigned as Head of Safety at the Center for AI Standards and Innovation and returned to lead the Alignment Research Center, writing on August 4-5 that mainstream alignment has a 20-30% chance of failing before broadly superhuman AI and giving ARC's mechanistic explanation work roughly 10% odds of meeting its most ambitious goals; Miles Brundage commented: 'And to think, the White House tried so hard to make it a hospitable work environment'.
Paul Christiano's departure from government AI safety
Paul Christiano resigns from US government's CAISI, returns to lead ARC as executive director.
Christiano is moving from a US government AI safety role to lead independent mechanistic interpretability research, citing his view that the safety community is undervaluing that approach. His stated probability estimates for mainstream alignment failure carry weight given his position in the field.
The full picture
Paul Christiano has resigned from his role as Head of Safety at the Center for AI Standards and Innovation (CAISI) and returned to the Alignment Research Center (ARC) as executive director. In a post explaining the move, Christiano describes ARC's agenda as finding mechanistic explanations for neural network behavior, which he presents as a backup plan if mainstream alignment techniques fail before broadly superhuman AI arrives. He estimates a 20-30% chance that current alignment approaches, including generalization shaping, malicious behavior prevention, and misalignment detection, break down before that point. He gives ARC roughly a 10% chance of achieving its most ambitious goals before superhuman AI makes human labor obsolete, and says success would cut AI takeover risk by a meaningful amount. ARC is hiring an automation lead and chief of staff, with a researcher round planned within months.
How it developed
Miles Brundage comments publicly on Christiano's departure from CAISI and return to ARC.
Christiano posts on the Alignment Forum announcing his return to ARC as executive director and explaining the research agenda.
Sources
Want this in your inbox?
I send a short email each morning with the stories that moved. If you would rather just read here, that works too.
Subscribe free