One path through the machine.
The core path is 12 lessons, about 3 hours. You follow one sentence through a real language model and take it apart as you go.
0 of 12 done
- Press EnterFree· 20 minWhat is an AI doing when it answers you?
- TokensIn the course· 14 minWhat does a model actually read?
- EmbeddingsIn the course· 14 minWhere does meaning live?
- One neuronIn the course· 11 minWhat can a single unit decide?
- Gradient descentIn the course· 14 minHow does it get less wrong?
- AttentionIn the course· 16 minHow does the last token find the one it needs?
- The server and the rackIn the course· 14 minWhat is inside the box, and where does the heat go?
- PretrainingIn the course· 13 minWhat is it trained to do?
- Fine-tuningIn the course· 14 minHow did it learn to be helpful?
- HallucinationIn the course· 12 minWhy does it make things up?
- The context windowIn the course· 14 minWhat can it see right now?
- Agents and memoryIn the course· 13 minHow does it work for an hour by itself?
Go deeper
The whole machine.
8 parts, 35 lessons, about 7 hours. The core path runs through all of it, and every other lesson takes one part further. They are all part of the course.
12 lessons open23 on the way
Part 1 · Press Enter
What happens between your message and its reply?
Play the model's own game, then follow one sentence through the whole machine.
Part 2 · Words into numbers
How can a machine that only does arithmetic hold meaning?
Cut text into tokens, walk a real map of embeddings, then search by meaning instead of by keyword.
- TokensCore pathWhat does a model actually read?In the course
- EmbeddingsCore pathWhere does meaning live?In the course
- Search by meaningHow does it find the right page in a million?On the way (on the way)
Part 3 · A network that learns
Nobody programmed it. So who set the numbers?
Wire a neuron by hand, roll downhill, then watch small networks teach themselves in your browser.
- One neuronCore pathWhat can a single unit decide?In the course
- Gradient descentCore pathHow does it get less wrong?In the course
- Deep networksWhy do layers matter?On the way (on the way)
- OverfittingHow do you know it really learned?On the way (on the way)
- Three ways to learnWhat are the kinds of machine learning?On the way (on the way)
- Reinforcement learningHow does a machine learn to win a game?On the way (on the way)
Part 4 · Attention and the transformer
How does a word know which other words matter?
Open a real model: delete its attention heads, switch off its memory, scramble its sense of order.
- AttentionCore pathHow does the last token find the one it needs?In the course
- The feed-forward layerWhere does it keep what it knows?On the way (on the way)
- Word orderHow does it know dog bites man from man bites dog?On the way (on the way)
- InterpretabilityCan anyone read what a model is thinking?On the way (on the way)
Part 5 · What it runs on
All of that is arithmetic. What is doing it, and what does it take from the world?
Take a chip, a server and a data center apart piece by piece, then run the power and the water yourself.
- The chipWhy does AI need its own kind of chip?On the way (on the way)
- The server and the rackCore pathWhat is inside the box, and where does the heat go?In the course
- The data centerWhat is the building for?On the way (on the way)
- Power and waterWhere does the electricity come from, and what does one answer cost the world?On the way (on the way)
- From sand to chipWho can make these, and why so few?On the way (on the way)
Part 6 · How it was made
What did it read, what did that cost, and why is it sometimes so sure and so wrong?
Train a tiny language model from noise, price a prompt, and catch a model bluffing.
- PretrainingCore pathWhat is it trained to do?In the course
- What it costsWhy is this so expensive to build and to run?On the way (on the way)
- Fine-tuningCore pathHow did it learn to be helpful?In the course
- HallucinationCore pathWhy does it make things up?In the course
- Evals and benchmarksHow do you test a machine that passed every test?On the way (on the way)
Part 7 · From model to assistant
How does next-word prediction become something that does work?
Fill a context window until it spills, be the app that runs a tool, and run an agent loop by hand.
- The context windowCore pathWhat can it see right now?In the course
- PromptingWhy does wording change the answer?On the way (on the way)
- ReasoningWhat changes when it reasons first?On the way (on the way)
- ToolsHow does a text predictor do things?On the way (on the way)
- Agents and memoryCore pathHow does it work for an hour by itself?In the course
Part 8 · Beyond the chat box
What else can this idea do, and where is it going?
Pull a picture out of noise, shrink a model until it breaks, and test your own map of the field.
- DiffusionHow does the same idea make images?On the way (on the way)
- Multimodal modelsHow does one machine take in pictures and sound?On the way (on the way)
- Video and world modelsCan a model learn physics by watching?On the way (on the way)
- Small, large, open, closedHow small can useful get?On the way (on the way)
- Alignment and safetyWhy is it hard to make a machine want what you meant?On the way (on the way)
- The whole mapWhat do you know now, and how do you keep up?On the way (on the way)
Start where the machine starts.
What is an AI doing when it answers you?
Start: Press Enter, free