Inside the Machine

One path through the machine.

The core path is 12 lessons, about 3 hours. You follow one sentence through a real language model and take it apart as you go.

0 of 12 done

Start: Press Enter, freeStop 1 of 12. About 20 minutes.
  1. Press EnterFree· 20 minWhat is an AI doing when it answers you?
  2. TokensIn the course· 14 minWhat does a model actually read?
  3. EmbeddingsIn the course· 14 minWhere does meaning live?
  4. One neuronIn the course· 11 minWhat can a single unit decide?
  5. Gradient descentIn the course· 14 minHow does it get less wrong?
  6. AttentionIn the course· 16 minHow does the last token find the one it needs?
  7. The server and the rackIn the course· 14 minWhat is inside the box, and where does the heat go?
  8. PretrainingIn the course· 13 minWhat is it trained to do?
  9. Fine-tuningIn the course· 14 minHow did it learn to be helpful?
  10. HallucinationIn the course· 12 minWhy does it make things up?
  11. The context windowIn the course· 14 minWhat can it see right now?
  12. Agents and memoryIn the course· 13 minHow does it work for an hour by itself?

Go deeper

The whole machine.

8 parts, 35 lessons, about 7 hours. The core path runs through all of it, and every other lesson takes one part further. They are all part of the course.

12 lessons open23 on the way

  1. Part 1 · Press Enter

    What happens between your message and its reply?

    Play the model's own game, then follow one sentence through the whole machine.

  2. Part 2 · Words into numbers

    How can a machine that only does arithmetic hold meaning?

    Cut text into tokens, walk a real map of embeddings, then search by meaning instead of by keyword.

  3. Part 3 · A network that learns

    Nobody programmed it. So who set the numbers?

    Wire a neuron by hand, roll downhill, then watch small networks teach themselves in your browser.

  4. Part 4 · Attention and the transformer

    How does a word know which other words matter?

    Open a real model: delete its attention heads, switch off its memory, scramble its sense of order.

  5. Part 5 · What it runs on

    All of that is arithmetic. What is doing it, and what does it take from the world?

    Take a chip, a server and a data center apart piece by piece, then run the power and the water yourself.

  6. Part 6 · How it was made

    What did it read, what did that cost, and why is it sometimes so sure and so wrong?

    Train a tiny language model from noise, price a prompt, and catch a model bluffing.

  7. Part 7 · From model to assistant

    How does next-word prediction become something that does work?

    Fill a context window until it spills, be the app that runs a tool, and run an agent loop by hand.

  8. Part 8 · Beyond the chat box

    What else can this idea do, and where is it going?

    Pull a picture out of noise, shrink a model until it breaks, and test your own map of the field.

    • DiffusionHow does the same idea make images?On the way (on the way)
    • Multimodal modelsHow does one machine take in pictures and sound?On the way (on the way)
    • Video and world modelsCan a model learn physics by watching?On the way (on the way)
    • Small, large, open, closedHow small can useful get?On the way (on the way)
    • Alignment and safetyWhy is it hard to make a machine want what you meant?On the way (on the way)
    • The whole mapWhat do you know now, and how do you keep up?On the way (on the way)

Start where the machine starts.

What is an AI doing when it answers you?

Start: Press Enter, free