The Information Machine
ConcludedRan 8 days1 sourceupdated 5 Nov 2026

Zvi, Habryka, Yudkowsky Rebut Alexander's Case Against Unilateral AI Pause

The gist

The debate concerns what AI safety researchers should concretely do, whether to stay inside labs, quit publicly, or push for coordinated pause, and the arguments bear directly on how the safety community allocates effort. The substantive disputes over China, superforecaster reliability, and inside-vs-outside advocacy have practical implications for strategy choices being made now.

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

The full picture

Scott Alexander published a post arguing against SURA (Stop Unilaterally and Raise Awareness), contending that unilateral stoppage of AI development undermines Plan C fallback strategies even if a global pause fails. Zvi Mowshowitz responded that most of Alexander's core claims are false, overstated, or apply only to extreme variants of SURA rather than reasonable ones. Zvi argued that prominent AI safety researchers who quit major labs and went public, such as Jacob Coxon and Daniel Kokotajlo, shifted public and political debate in ways internal advocates rarely achieve. On the China question, Zvi argued the point that a Western pause would be futile is overstated because Chinese frontier models have relied heavily on distilling Claude and GPT. Zvi also argued that AI superforecasters have an 'atrocious track record' on core AI development milestones and should not serve as a primary input for strategic decisions. Oliver Habryka contended that roughly 75% of rationalist and EA community efforts have gone toward gaining power at or starting AI labs, and that DC insiders actively suppressed open public advocacy about existential risk. Habryka and Zvi argued that DC policy coalitions supporting chip export controls were deliberately constructed around cover motivations disconnected from actual existential risk concerns. Eliezer Yudkowsky pressed Alexander to state what policy he would advocate if ASI is genuinely lethal, and Alexander acknowledged he would be much less confident in his conclusions under that premise. TsviBT contended that Anthropic has not made a serious, well-resourced effort toward a global pause despite having billions in capital and thousands of employees, and proposed middle-ground options such as doing frontier alignment research without releasing leading models and aggressively negotiating a conditional stop with other labs. Daniel Kokotajlo argued that corporate incentives are a deeply corrupting influence and individuals should identify which corrupting force bears most on them personally.

How it developed
5 November 2026

DeepSeek engineer Gao Xi says company will keep releasing model weights to hasten AGI despite higher safety risks

2 November 2026

Zvi published rebuttal to Scott Alexander's 'Against SURA,' with Habryka, Yudkowsky, Vaniver, Kokotajlo, and TsviBT contributing to the debate.

30 October 2026

Kokotajlo criticizes Amodei for calling for pacing without committing to unilateral action; Warren raises public market risk concerns

29 October 2026

Altman says the world may need to cap how smart a single AI model is allowed to get

Sources

Get the daily email

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

See the latest edition · Privacy