Glossary · Phase 3 · Machine learning
Early stopping
Stopping training once the validation score stops improving, and keeping the best version so far.
Boosting needs care not to overfit. Each new tree chases the leftover errors, and some of those are noise. Use a small learning rate. Use early stopping, too: keep checking the score on validation data, and stop adding trees when it stops improving.
Try it now
Same data and depth. Compare one tree's jagged boxes with the smoother boundary from averaging 25.
Learn it properly
Random forests and boosting · Many weak models voting beat one strong one. About 30 minutes.
Other terms from the same lesson
- bagging: Training many models, each on a random resample of the data, and averaging their answers. The averaging cancels out much of each model's random error.
- ensemble: Several models whose answers are combined, by voting or averaging. Together they usually beat any one of them.
- gradient boosting: Adding small decision trees one at a time, each fixing the mistakes left by the ones before.
- random forest: Many decision trees trained on different samples, voting together.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.