Glossary · Phase 3 · Machine learning
Regularization
Gently penalizing a model for being too complicated, to stop it overfitting.
Try it now
Slide the penalty and watch the error on new points fall, then rise again when the penalty is too strong.
Learn it properly
Regularization · Penalize complexity so models generalize. About 30 minutes.
Other terms from the same lesson
- elastic net: Regularization that mixes the ridge and lasso penalties.
- lasso: Regularization that adds the plain size of every weight to the loss. It pushes unhelpful weights to exactly zero, so it also picks features.
- ridge regression: Regularization that adds the squared size of every weight to the loss, so weights stay small but rarely reach exactly zero.
- weight decay: Shrinking every weight a tiny bit on each step, so the model keeps its weights small and is less likely to overfit.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.