Glossary · Phase 3 · Machine learning
Random forest
Many decision trees trained on different samples, voting together.
A random forest adds a second source of difference. At each split, a tree may only consider a random handful of the features. Sometimes the dominant feature isn't on the list. The tree has to find the best question among the others. The trees end up asking different questions, so they disagree more usefully.
Try it now
Same data and depth. Compare one tree's jagged boxes with the smoother boundary from averaging 25.
Learn it properly
Random forests and boosting · Many weak models voting beat one strong one. About 30 minutes.
Other terms from the same lesson
- bagging: Training many models, each on a random resample of the data, and averaging their answers. The averaging cancels out much of each model's random error.
- early stopping: Stopping training once the validation score stops improving, and keeping the best version so far.
- ensemble: Several models whose answers are combined, by voting or averaging. Together they usually beat any one of them.
- gradient boosting: Adding small decision trees one at a time, each fixing the mistakes left by the ones before.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.