Glossary · Phase 6 · Modern LLM engineering
Interpretability
Research into what is actually happening inside a model's numbers.
Try it now
A picture of attention is only a hypothesis. Switching parts off is what tests it.
Learn it properly
Interpretability · Looking inside models to see what they compute. About 40 minutes.
Other terms from the same lesson
- ablation: Switching off one part of a model, such as an attention head, to see what changes. A big change means that part mattered.
- logit lens: Reading a model's in-progress guess at each layer by passing the hidden state straight to the output layer.
- steering: Changing a model's behavior by adding a direction to its hidden states while it runs, without retraining it.
- superposition: When a model packs more features than it has neurons by sharing each neuron across several of them, which makes single neurons hard to read.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.