Glossary · Phase 5 · Transformers
Lower-triangular
A grid with numbers on and below the diagonal and zeros above it. The causal mask has this shape: each token sees only itself and earlier ones.
Try it now
Each row is one word. With the mask on it can't see anything to its right, and its 100% is shared among the words left.
Learn it properly
Causal masking · Stop tokens from seeing the future. About 25 minutes.
Other terms from the same lesson
- causal mask: A rule that stops each token looking at the tokens after it, so the model can't peek at the answer.
- teacher forcing: Training on real text by predicting every next token at once from the true earlier tokens, not the model's own guesses.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.