Glossary · Phase 3 · Machine learning
Calibration
Whether a model's probabilities can be taken at face value: of the customers it gives 30%, about 30% really churn.
Try it now
The curve is every threshold at once, and AUC is the area under it. Lower the model's skill to 0 and it falls onto the diagonal: no better than chance.
Learn it properly
ROC curves and AUC · Judge a classifier across every threshold. About 30 minutes.
Other terms from the same lesson
- AUC: Area under the ROC curve: the chance the model scores a random true case above a random false one. 1 is perfect, 0.5 is guessing.
- average precision: The area under the precision-recall curve: one number for how well a model finds a rare class.
- false positive rate: The share of real negatives the model wrongly flags, such as the share of loyal customers it says will churn.
- precision-recall curve: A chart of precision against recall as the threshold moves. More telling than the ROC curve when the thing you're looking for is rare.
- ROC curve: A chart of how a classifier trades catching more true cases against raising more false alarms.
- true positive rate: The share of real positives the model catches, such as the share of churners it flags. Another name for recall.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.