The Information Machine
Learn · Guide 13 of 13

Pausing or pacing AI development

This guide explains what pausing or pacing frontier AI development would involve, the main arguments for and against it, and the proposals and groups pushing for it.

A coordinated pause or pacing: what it would mean#

Pausing frontier AI means stopping, for a time, the training of models more powerful than today's best; pacing lets that training continue but limits how fast it advances.1 Unlike most regulation, which governs how AI is used, both target the development of the strongest models, not AI research in general.213

A pause can be unilateral, with one lab stopping on its own, or coordinated, with all major developers stopping together; it can also be conditional, applying only if a model crosses a danger threshold before safeguards are ready.13 One lab can adopt such a condition alone: Anthropic's responsible scaling policy, first released in September 2023, said the company would not train or deploy models without adequate safeguards.4 Pacing gates each step, for example with stronger safeguards at each capability threshold or a cap on how fast training compute grows.41 A central practical problem is verification: knowing a rival has not kept training in secret, perhaps by monitoring the specialized chips it needs.5 Local data center bans are a separate issue.6

In March 2023, the Future of Life Institute, a group focused on technological risks, published an open letter.789 It asked all labs to pause, for at least six months, training anything more powerful than GPT-4, which OpenAI had released that month.17 The pause was to be 'public and verifiable', with a government moratorium, or mandated halt, if labs did not act quickly.1 Signers included AI pioneer Yoshua Bengio, Elon Musk and Apple co-founder Steve Wozniak.89 Wired reported that the letter did not say how such a halt could be verified.8

The case for a pause#

Some researchers and public figures argue that work on the most powerful AI systems should pause or slow down until safety research and oversight catch up.1011

In "Managing extreme AI risks amid rapid progress," a consensus paper first posted in October 2023 and revised in May 2024, the authors wrote that AI "is progressing rapidly" while "AI safety research is lagging."12 In October 2025, more than 850 people, including the leading AI researchers Yoshua Bengio, Geoffrey Hinton and Stuart Russell, signed a public statement calling for a ban on superintelligence until there is strong public support for it and a scientific consensus that it can be built and controlled safely (see the proposals section).10 In September 2026, Jacob Coxon, a researcher who quit Anthropic, wrote of Anthropic and OpenAI: "Neither company is acting responsibly. They are racing straight to self-improving superintelligence."1314 Later in September 2026, Anthropic's CEO Dario Amodei argued for pacing, slowing rather than stopping, writing that "we must slow the pace at which we improve the capabilities of AI models" and "we must make wise use of the time we gain."151114 Responding to Amodei's proposal in September 2026, Coxon told the BBC that a slowdown would need to be co-ordinated with China to avoid "a race at an international scale."16

In 1974, Paul Berg and colleagues published a letter in Science asking for a moratorium on recombinant DNA research, which joins genes from different species, until its hazards could be assessed.1718 Scientists feared the technique could turn harmless microbes into human pathogens.17 Many scientists protested, yet this voluntary moratorium, the first in the history of science, was universally observed.17

The case against, including the competition-with-China argument#

Critics of the case for a pause argue that halting frontier AI would be hard to verify, would hand the lead to countries that keep going, and would delay benefits AI already brings.1920

A pause agreed between countries only holds if each can verify that the others comply, yet training a safe model can look nearly identical to training an unsafe one.215 Critics also point to a race dynamic: each country might be better off if all paused, yet each has a reason to break ranks and keep building.19 After the Future of Life Institute's October 2025 statement calling for a halt on superintelligence until there is broad scientific consensus that it can be built safely, with strong public buy-in, the American Enterprise Institute (AEI) argued that a pause by the US alone would "delay innovation, empower bureaucracies, and hand an enormous advantage to China."19

When the Future of Life Institute called for a six-month pause on training more powerful AI models, Yann LeCun and Andrew Ng publicly argued against it.22 LeCun said he supported regulating products that reach people but saw no point in regulating research.22 Ng said harms such as bias and concentration of power are real, and both rejected a research halt as the answer to them, which shows these critics accept the risks but dispute the remedy.2220

Concrete proposals and who backs them#

Proposals to pause or pace AI range from labs holding back when tests find dangerous abilities, to international limits on AI training, to a ban on building superintelligence until strict conditions are met.232410

The mildest are responsible scaling policies (RSPs): a developer states which AI abilities it can handle safely, and when it would stop until its protections improve.23 They rest on if-then commitments: if evaluations show a dangerous ability, safeguards agreed in advance must be in place, or the work is delayed.2523 METR, the AI evaluation group that proposed RSPs, wrote that they were a first step for labs, not a substitute for regulation.2623 A paper posted by Peter Barnett proposes the middle option: a coalition led by the United States and China that would restrict the scale of AI training.24 Its authors wrote in November 2025 that the political will for such an agreement did not yet exist.27 At the strongest end, CNBC reported that in October 2025 more than 850 people, including AI pioneers Yoshua Bengio and Geoffrey Hinton, signed a statement calling for a ban on superintelligence until it has strong public support and a scientific consensus that it can be built and controlled safely.10

Anthropic's Responsible Scaling Policy, first released in September 2023, used this approach: if a model gained biological science abilities that could help create dangerous weapons, the policy called for stricter safeguards against misuse and theft of its weights.2829

Keep going
Sources (29)
  1. 127 lesswrong.com
  2. Toward a Global Regime for Compute Governance: Building the Pause Button arxiv.org
  3. Common Elements of Frontier AI Safety Policies metr.org
  4. Announcing our updated Responsible Scaling Policy anthropic.com
  5. What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring arxiv.org
  6. Proposed Data Center Ordinance & Moratorium miamicountyin.gov
  7. Fearing “loss of control,” AI critics call for 6-month pause in AI development arstechnica.com
  8. In Sudden Alarm, Tech Doyens Call for a Pause on ChatGPT wired.com
  9. Letter signed by Elon Musk demanding AI research pause sparks controversy theguardian.com
  10. Hundreds of public figures, including Apple co-founder Steve Wozniak and Virgin's Richard Branson urge AI ‘superintelligence’ ban cnbc.com
  11. Anthropic boss Dario Amodei calls for AI development to slow down bbc.com
  12. Managing extreme AI risks amid rapid progress arxiv.org
  13. Experts weigh in as researcher says AI has more than 10% chance of 'killing all humans' cnbc.com
  14. ‘We must slow the pace’: CEO of Anthropic calls for an AI slowdown theguardian.com
  15. Anthropic CEO outlines plan to slow AI development | TechCrunch techcrunch.com
  16. AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC bbc.com
  17. Recombinant DNA Technologies and Researchers' Responsibilities, 1973-1980 profiles.nlm.nih.gov
  18. Potential Biohazards of Recombinant DNA Molecules [Letter to the Editor of Science] profiles.nlm.nih.gov
  19. Are we Doomed to an AI Race? Why Self-Interest Could Drive Countries Towards a Moratorium on Superintelligence arxiv.org
  20. Yann LeCun and Andrew Ng: Why the 6-month AI Pause is a Bad Idea digital-skills-jobs.europa.eu
  21. AI ALIGNMENT FORUM alignmentforum.org
  22. LESSWRONG lesswrong.com
  23. Responsible Scaling Policies (RSPs) metr.org
  24. An International Agreement to Prevent the Premature Creation of Artificial Superintelligence arxiv.org
  25. If-Then Commitments for AI Risk Reduction carnegieendowment.org
  26. 2023 Year In Review metr.org
  27. An International Agreement to Prevent the Premature Creation of Artificial Superintelligence arxiv.org
  28. Anthropic’s Responsible Scaling Policy anthropic.com
  29. Responsible Scaling Policy Version 3.0 anthropic.com