The Information Machine
Learn · Series 3 of 3

Safety and governance

How AI can fail, how people check it, and who decides the rules: the risks, the testing, the security, the long-term questions, and the laws and politics around them. The policy news itself is in the stories I follow.

  1. AI safety: a map of the field

    This guide maps the main kinds of AI risk, open questions about AI systems themselves, and the research and policy work that addresses them, with links to deeper guides.

    In this guide

    1. What AI safety covers
    2. Misuse and dual-use capabilities
    3. A different kind of question
    Read the guide →

  2. When AI doesn't do what we meant

    This guide explains what it means for an AI model to do what its developers intend, and the ways models fall short, from gaming rewards to hiding their abilities.

    In this guide

    1. Alignment
    2. Reward hacking
    3. Sycophancy
    4. Emergent misalignment
    5. Situational awareness
    6. Scheming and alignment faking
    7. Sandbagging
    Read the guide →

  3. Checking AI: testing, audits and looking inside models

    How researchers test, red-team, audit and look inside AI models, read their written reasoning, and try to prove their behavior, plus what each method can and cannot tell us.

    In this guide

    1. Evaluations
    2. Red teaming
    3. Audits
    4. Interpretability
    5. Chain-of-thought monitoring and why it could be lost
    6. Formal verification
    Read the guide →

  4. AI security: attacks and defenses

    This guide explains how deployed AI models are guarded, how jailbreaks, prompt injection and sandbox escapes get around those guards, and how security flaws and user data are handled.

    In this guide

    1. Guardrails
    2. Jailbreaking
    3. Prompt injection
    4. Sandbox escape
    5. Vulnerability disclosure
    6. Data retention
    Read the guide →

  5. Superintelligence and long-term risk

    This guide explains what superintelligent AI would be, when and how quickly it might arrive, and the main arguments about whether people could lose control of it.

    In this guide

    1. Superintelligence
    2. AI timelines
    3. Recursive self-improvement
    4. Loss of control
    5. How likely is catastrophe? Existential risk, "P(doom)" estimates, and the main positions
    Read the guide →

  6. Governing AI: liability, regulation and geopolitics

    This guide explains who is responsible when AI causes harm, how the EU, US and China regulate AI, why data centers face local pushback, and how governments control AI chips and compute.

    In this guide

    1. Liability
    2. The EU AI Act
    3. The United States
    4. China's AI rules
    5. Content provenance and watermarking
    6. Data center bans and pauses
    7. Regulatory capture
    8. AI chip export controls
    9. Sovereign AI
    Read the guide →

  7. Pausing or pacing AI development

    This guide explains what pausing or pacing frontier AI development would involve, the main arguments for and against it, and the proposals and groups pushing for it.

    In this guide

    1. A coordinated pause or pacing
    2. The case for a pause
    3. The case against, including the competition-with-China argument
    4. Concrete proposals and who backs them
    Read the guide →

Get the daily email

The guides explain the ideas. The daily email follows what happens next.

One free email each morning, around 8 AM US Eastern.

See the latest edition · Privacy

Keep going
  • Foundations: how AI works: The core ideas behind AI: what a language model is, how it is trained, the chips and data centers it runs on, how models are compared, and how to get answers from one.
  • Agents: Models that take actions: how a model becomes an agent, how it uses tools and screens, why long tasks are hard, and what happens when an agent goes outside its instructions.
  • Glossary: the terms, in a sentence or two each
  • All guides