Glossary · Phase 5 · Transformers
Teacher forcing
Training on real text by predicting every next token at once from the true earlier tokens, not the model's own guesses.
Try it now
Each row is one word. With the mask on it can't see anything to its right, and its 100% is shared among the words left.
Learn it properly
Causal masking · Stop tokens from seeing the future. About 25 minutes.
Other terms from the same lesson
- causal mask: A rule that stops each token looking at the tokens after it, so the model can't peek at the answer.
- lower-triangular: A grid with numbers on and below the diagonal and zeros above it. The causal mask has this shape: each token sees only itself and earlier ones.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.