Base-rate fallacy
Forgetting how rare something is to begin with. A 95%-accurate fraud flag can still be wrong most of the time when fraud is rare.
What it means
The base rate fallacy is judging something by how accurate a test sounds, while forgetting how rare the thing is to begin with. The base rate is how common something is before you test for it. When it's low, even a good test produces more false alarms than real finds, simply because there are so many more chances to raise a false alarm.
An example: a 95% accurate fraud flag
customers = 100_000 fraud = customers * 0.01 # 1% of orders are fraud honest = customers - fraud caught = fraud * 0.95 # the flag catches 95% of fraud false_alarms = honest * 0.05 # and wrongly flags 5% of honest orders print(f"flagged: {caught + false_alarms:,.0f} orders") print(f"really fraud: {caught:,.0f}") print(f"chance a flagged order is fraud: {caught / (caught + false_alarms):.0%}")
flagged: 5,900 orders really fraud: 950 chance a flagged order is fraud: 16%
The flag sounds excellent, and it does catch 95% of fraud. But honest orders outnumber fraud 99 to 1, so 5% of them is far more than 95% of the fraud. Only about one flagged order in six is really fraud. Bayes' rule gives the same answer, and counting people is the easy way to see it.
Why it matters for AI
It's why precision can look terrible for a classifier with a high accuracy, and why scores from a balanced test set don't carry over to the real world. Before you trust a detector, whether it's for fraud, spam or churn, test it on data with the real mix, or work out what its precision will be at the real base rate.
Try it now
Rare things mostly produce false alarms, even with a good test. Raise how common it is and watch the answer jump.
Learn it properly
Conditional probability and Bayes · Update beliefs when new evidence arrives. About 30 minutes.
Other terms from the same lesson
- Bayes' rule: A way to update how likely something is when new evidence arrives.
- conditional probability: How likely something is once you know something else is true, like rain given dark clouds.
- posterior: Your updated belief after seeing the evidence. Bayes' rule turns the prior into the posterior.
- prior: What you believe about a chance before you see new evidence, such as 1% of payments being fraud.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.