Conditional probability and Bayes' rule
Why this matters
A chance changes when you learn something. The chance of rain is one number; the chance of rain given that the sky is black is another. That "given" is conditional probability, and it's everywhere in machine learning: a language model predicts the next word given the words so far, and a spam filter scores a message given its words. A classic trap comes with it: the chance a test is positive given you're sick is not the chance you're sick given the test is positive, and confusing them can be badly wrong. Bayes' rule is the bridge between the two. This lesson builds both from a simple table of counts.
- Find P(A | B) from a two-way table: keep only the B column, then take the share
- See that P(A | B) and P(B | A) are different numbers, often very different
- Use Bayes' rule, drawn as paths on a tree, to turn one into the other, as for a test with false alarms
- Read a classifier's results table both ways, precision and recall, in NumPy and with PyTorch
The math: restrict, then take the share
Purpose. Update a chance when something is known.
A sign classifier was tested on 100 signs. The table counts what each sign was (rows) and what it was called (columns):
| blank | called tent | called tarp |
|---|---|---|
| tent | 40 | 10 |
| tarp | 5 | 45 |
"Given it was called tent" means: look only at that column, 45 signs. Of those, 40 were tents:
In plain words: shrink the world to the cases where B happened, then ask what share of those have A.