Hinton, Bengio Paper Calls for AI R&D Automation Oversight

The paper 'What if Automating AI R&D Triggers an Intelligence Explosion?' co-authored by Geoffrey Hinton, Yoshua Bengio, Jakub Pachocki, Jack Clark, Hilary Greaves, Tom Davidson, and others calls on policymakers to demand transparency from AI companies about how far they have already automated their model development processes and how close they are to recursive self-improvement (RSI). The Wall Street Journal covered the paper under the headline 'Top AI Researchers Call for Urgent Oversight of Self-Improving Systems.' The paper also proposes specific policy interventions to respond if automating AI R&D begins triggering an intelligence explosion. Separately, advocates are pressing Congress to ban RSI, defined as AI systems autonomously building the next generation of AI, citing OpenAI's stated timeline of March 2028 and Anthropic employees' claimed 2027 target. Support for such a ban spans Bernie Sanders, Steve Bannon, Yoshua Bengio, Ezra Klein, and multiple state attorneys general. A METR report is cited as evidence that those timelines are credible. Advocates argue the ban need not be permanent but frame it as urgent, noting that the threshold may be reached before the next U.S. election. In parallel, AI safety researchers debate what an intelligence explosion would look like: Paul Christiano defines slow takeoff as a world where a 4-year output doubling precedes a 1-year doubling; Eliezer Yudkowsky expects growth to stay gradual and then collapse catastrophically in a very short window; Forethought identifies three types of intelligence explosion; Arvind Narayanan argues any RSI will be 'jagged,' with continued progress on verifiable tasks but persistent difficulty on unverifiable ones; and Richard Ngo expects divergence between measured capabilities and real-world impacts to persist.