The Information Machine
Learn · Series 1 of 3

Foundations: how AI works

The core ideas behind AI: what a language model is, how it is trained, the chips and data centers it runs on, how models are compared, and how to get answers from one.

  1. How a language model works

    A look inside a language model: what it is as a file, how it splits text into tokens, turns them into numbers, handles long inputs, and uses only some parameters per word.

    In this guide

    1. Model weights
    2. Tokens
    3. Embeddings
    4. The context window
    5. Mixture of experts
    Read the guide →

  2. How AI models are trained

    Follow a model from raw text to helpful assistant: instruction tuning, fine-tuning for specific jobs, learning from scores and human feedback, and copying one model's skills into another.

    In this guide

    1. Pretraining and post-training
    2. Fine-tuning
    3. Reinforcement learning
    4. Distillation
    Read the guide →

  3. Compute: the chips, clouds and data centers behind AI

    This guide explains why AI needs so much specialized hardware, how training and running models cost money, and where the chips, data centers and rented cloud capacity come from.

    In this guide

    1. What compute is and why it scales
    2. GPUs and accelerators
    3. Data centers and power
    4. Training versus inference
    5. Neoclouds
    Read the guide →

  4. Frontier models, open models, and how they are compared

    This guide explains what makes a model frontier, what changes when a model's weights are released openly, and how much you can rely on benchmark scores to compare models.

    In this guide

    1. What counts as a frontier model
    2. Open-weight models
    3. Benchmarks and their limits
    Read the guide →

  5. Getting answers from a model: prompts, reasoning and routing

    This guide explains how models receive standing instructions, why writing out steps can improve answers, what extra thinking time buys you, and how products pick which model responds.

    In this guide

    1. The system prompt
    2. Chain-of-thought
    3. Reasoning effort
    4. Model routing
    Read the guide →

Get the daily email

The guides explain the ideas. The daily email follows what happens next.

One free email each morning, around 8 AM US Eastern.

See the latest edition · Privacy

Keep going
  • Agents: Models that take actions: how a model becomes an agent, how it uses tools and screens, why long tasks are hard, and what happens when an agent goes outside its instructions.
  • Safety and governance: How AI can fail, how people check it, and who decides the rules: the risks, the testing, the security, the long-term questions, and the laws and politics around them. The policy news itself is in the stories I follow.
  • Glossary: the terms, in a sentence or two each
  • All guides