Glossary · Phase 3 · Machine learning
Gradient boosting
Adding small decision trees one at a time, each fixing the mistakes left by the ones before.
Try it now
Same data and depth. Compare one tree's jagged boxes with the smoother boundary from averaging 25.
Learn it properly
Random forests and boosting · Many weak models voting beat one strong one. About 30 minutes.
Other terms from the same lesson
- bagging: Training many models, each on a random resample of the data, and averaging their answers. The averaging cancels out much of each model's random error.
- early stopping: Stopping training once the validation score stops improving, and keeping the best version so far.
- ensemble: Several models whose answers are combined, by voting or averaging. Together they usually beat any one of them.
- random forest: Many decision trees trained on different samples, voting together.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.