Glossary · Phase 5 · Transformers
Training loss
The model's average error on the examples it learns from. It should fall as training goes on.
Try it now
The same loop you've used since Phase 3. The model is what's new.
Learn it properly
Build a tiny GPT · Write and train a small GPT from scratch. About 45 minutes.
Other terms from the same lesson
- validation loss: The model's average error on held-back text it never trains on. If it rises while training loss keeps falling, the model is memorizing.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.