Probability basics
Events, outcomes and the rules for combining them.
Create a free account Try part of it here
Why it matters
A language model doesn't output a single word. Its raw output is a probability for every possible next word, and a word is then picked using those numbers. Classifiers output probabilities too. Evals estimate them, and A/B tests are built on them. If you work in marketing, you already think in probabilities: open rates, conversion rates, churn. What's often missing is the small set of rules that keeps that thinking straight. This lesson gives you those rules, plus a habit that catches mistakes: checking a formula by simulating it with random numbers.
What you'll be able to do
- Describe outcomes, events and probabilities between 0 and 1
- Use the complement, addition and independence rules
- Compute 'at least one' probabilities
- Check a probability formula by simulating it with random numbers
What it covers
- Outcomes, events, probabilities
- Rule 1: the complement
- Rule 3: independence (the "and" rule)
- "At least one"
- Check it by simulation
- Worked example: will anyone reply?
Outcomes, events, probabilities
Think of rolling a die. Before you roll, you know what can happen: 1, 2, 3, 4, 5 or 6. You don't know which one you'll get. That is the everyday setting for probability.
A few words name the parts:
- An experiment is anything with an uncertain result, like rolling a die or sending an email.
- An outcome is one possible result. The die shows 4. The person opens the email.
- An event is a group of outcomes you care about. "Rolled an even number" covers 2, 4 and 6. "Opened" is an event too.
The probability of an event is a number from 0 to 1 that says how likely it is. 0 means it never happens. 1 means it always happens. 0.5 is a coin toss. You can also say it as a percentage: 0.25 is 25%.
In math, "the probability of an event" is written as . The stands for "probability". The is a short name for the event, like a label on a jar. So reads "the probability of open is 0.25".
In plain words: means "how likely A is", written as a number from 0 to 1.
What does 0.25 actually mean? One useful reading: over many sends, about 25% get opened. This is called the frequency view. It's exactly what a simulation measures, as you'll see later.
The probabilities of all the possible outcomes add up to 1. A fair die gives each face 1/6, and six sixths make 1. Something has to happen. For example, "even" covers 3 of the 6 faces, so the chance of rolling an even number is 3/6, which is 0.5.
Show the answer
2/6, about 0.33
Only 5 and 6 are above 4. That's 2 of the 6 equally likely faces, so 2/6.
Rule 1: the complement
The complement of an event is everything else: the event not happening. If 25% of people open your email, the other 75% don't.
In plain words: the chance something doesn't happen is 1 minus the chance it does.
A worked example: a 25% open rate means . So the chance of not opening is 1 − 0.25 = 0.75. Three in four people don't open.
The same sum in Python:
p_open = 0.25 print(1 - p_open)
It prints 0.75. This rule looks almost too simple to matter. Later in this lesson it turns a hard question into an easy one.
Rule 2: addition (the "or" rule)
Picture a bakery with 100 customers one morning. 60 buy bread. 30 buy milk. 20 of them buy both. How many bought bread or milk?
Adding 60 + 30 gives 90, but that's wrong. The 20 who bought both were counted twice: once with the bread buyers and once with the milk buyers. Take them off once and you get 70.
That is the addition rule. Here, "A or B" means A, or B, or both.
In plain words: add the two chances, then subtract the overlap so it isn't counted twice.
As probabilities, the bakery sum is 0.60 + 0.30 − 0.20 = 0.70. So 70% of customers bought bread or milk.
p_bread = 0.60 p_milk = 0.30 p_both = 0.20 print(round(p_bread + p_milk - p_both, 2))
It prints 0.7. The round(..., 2) keeps two decimal places. Computers store most decimals with tiny errors, and rounding hides them.
Go deeper When there's no overlap
Some events can't happen together. One roll of a die can't be both a 1 and a 6. Such events are called mutually exclusive. Their overlap is 0, so the rule shrinks to plain adding. The chance of a 1 or a 6 is 1/6 + 1/6 = 2/6. Adding without subtracting is only safe in this case.
Show the answer
0.8
Add the two and subtract the overlap: 0.50 + 0.40 − 0.10 = 0.80. Plain adding counts the 10% who like both twice.
Rule 3: independence (the "and" rule)
Two events are independent if one happening doesn't change the chance of the other. Flip a coin and roll a die. Getting heads tells you nothing about the die.
For independent events, and only then, the chance that both happen is the two chances multiplied:
In plain words: if two things don't affect each other, the chance of both is one chance times the other. The dot means "times".
A worked example: heads has probability 0.5. A six has probability 1/6, about 0.167. So heads and a six together has probability 0.5 × 1/6 = 1/12. That's about 0.083, or 1 time in 12.
p_heads = 0.5 p_six = 1 / 6 print(round(p_heads * p_six, 3))
It prints 0.083.
Independence is an assumption, so always ask whether it makes sense. Two different customers opening the same newsletter are roughly independent. One customer opening and the same customer clicking are not. You can't click a link in an email you never opened. Events that depend on each other are the topic of the next lesson.
Go deeper Why multiplying works
Say 100 people get an email and 30% open it, so 30 people open. Say 60% of people read on mobile, and device has nothing to do with opening. Then 60% of those 30 openers are on mobile: 18 people. That's 18 out of 100, or 0.18, which is 0.30 × 0.60. Multiplying is "taking a share of a share".
If the two were linked (say mobile readers opened less often), the share of openers on mobile wouldn't be 60%. Then multiplying would give the wrong answer.
Show the answer
Two strangers in different cities each flipping their own coin
One stranger's coin can't affect the other's. In the other pairs, one event changes the chance of the other, so you can't multiply.
"At least one"
Here's a question that comes up all the time. You have 5 leads. Each one converts with probability 0.1, and they don't affect each other. What's the chance that at least one converts?
The direct route is painful. You'd add up "exactly one converts", "exactly two", and so on up to five. The complement is much easier. "At least one converts" has a single opposite: "none converts".
- Each lead fails with probability 1 − 0.1 = 0.9 (rule 1).
- All five fail with probability 0.9 × 0.9 × 0.9 × 0.9 × 0.9, about 0.59 (rule 3).
- So at least one converts with probability 1 − 0.59, about 0.41 (rule 1 again).
To write this as a formula, call the chance that one try succeeds . Call the number of tries .
In plain words: work out the chance that every try fails, then take that away from 1. The small raised means "multiply by itself times". The wavy equals sign means "is about".
In Python, ** means "to the power of":
p = 0.1 # chance one lead converts n = 5 # number of leads print(round(1 - (1 - p) ** n, 2))
It prints 0.41.
Notice that 5 tries at 20% each don't add up to a sure thing. The chance of at least one success is 1 − 0.8⁵, about 0.67, not 100%.
The same formula explains why small error rates pile up. An AI agent that makes 20 tool calls, each with a 2% chance of going wrong, fails somewhere about a third of the time.
Go deeper The long way around, for two tries
With two leads at , you can count every case. Exactly one converts in two ways: first yes and second no (0.1 × 0.9 = 0.09), or first no and second yes (another 0.09). Both convert with probability 0.1 × 0.1 = 0.01. The total is 0.09 + 0.09 + 0.01 = 0.19.
The shortcut gives 1 − 0.9² = 1 − 0.81 = 0.19. Same answer, far less work. The long way gets much longer as the number of tries grows.
Work out "none" first, then take it away from 1. Note that it's far less than just adding the chances.
print(round(1 - 0.5 ** 3, 3))
Show the answer
0.875
All tails has probability 0.5 × 0.5 × 0.5 = 0.125. At least one heads is the complement: 1 − 0.125 = 0.875.
Check it by simulation
Probability mistakes are easy to make and hard to spot. Here's a habit that catches many of them. Use random numbers to run the experiment thousands of times, then count what happened. If the share you count matches the formula, the formula (and your understanding) is right.
The trick works for any yes/no event. rng.random() gives a number anywhere between 0 and 1, with every value equally likely. Think of a spinner that can stop anywhere on a ruler from 0 to
- It stops below 0.1 about 10% of the time. So "the number is below 0.1" behaves like a lead
that converts with probability 0.1.
The code below plays out 100,000 rounds of the 5-leads question. Each round is one row of 5 random numbers.
import numpy as np rng = np.random.default_rng(42) trials = 100_000 converted = rng.random((trials, 5)) < 0.1 # 5 leads per trial, each converts with p = 0.1 at_least_one = converted.any(axis=1) print("simulated:", at_least_one.mean().round(4)) print("formula: ", round(1 - 0.9 ** 5, 4))
Here's what each step did:
np.random.default_rng(42)makes a random number generator. The 42 is a seed. It fixes the starting point, so you get the same "random" numbers on every run.rng.random((trials, 5)) < 0.1builds a table with 100,000 rows and 5 columns. Each cell is True where that lead converted..any(axis=1)asks of each row: is at least one cell True?.mean()of True/False values gives the share that are True, because True counts as 1 and False as 0.
The simulated share and the formula agree closely. Both are about 0.41.
Now the addition rule, with 10,000 email recipients. Each one opens with probability 0.30. Each one reads on mobile with probability 0.60. The two are drawn separately, so they are independent here.
import numpy as np rng = np.random.default_rng(0) opened = rng.random(10_000) < 0.30 on_mobile = rng.random(10_000) < 0.60 # independent of opening here either = opened | on_mobile both = opened & on_mobile print("P(open or mobile) simulated:", either.mean().round(3)) print("formula: 0.30 + 0.60 - 0.30 × 0.60 =", round(0.30 + 0.60 - 0.30 * 0.60, 3)) print("P(both):", both.mean().round(3))
In NumPy, | means "or" and & means "and", checked for each recipient. The formula line uses two rules at once. The overlap comes from rule 3: the events are independent, so the chance of both is 0.30 × 0.60 = 0.18. Then rule 2 gives 0.30 + 0.60 − 0.18 = 0.72.
The simulated "or" lands close to 0.72, and the simulated "both" lands close to 0.18. They won't match exactly. Random samples always wobble a little, and more trials make the wobble smaller.
Worked example: will anyone reply?
You text 4 friends to ask who's free tonight. Each one replies with probability 0.3, and they don't affect each other. What's the chance nobody replies? That at least one replies? That all four reply?
p = 0.3 # chance one friend replies n = 4 # number of friends none = (1 - p) ** n print("nobody replies:", round(none, 4)) print("at least one: ", round(1 - none, 4)) print("all four reply:", round(p ** n, 4))
Nobody replies means every friend stays silent: 0.7 × 0.7 × 0.7 × 0.7, about 0.24 (rules 1 and 3). At least one replies is the complement: 1 − 0.2401, about 0.76. All four reply is 0.3 × 0.3 × 0.3 × 0.3 = 0.0081, under 1%. "At least one" is likely here, while "all of them" is rare. Mixing up the two is a common mistake.
Show the answer
0.512
Each email is missed with probability 1 − 0.2 = 0.8. Missing all three is 0.8 × 0.8 × 0.8 = 0.8³ = 0.512.
Practice what you just read
- 7 code examples you can edit and run
- a practice exercise that checks your code
- a 5-question quiz
- 4 review cards for spaced repetition
Try a quiz question
When can you compute P(A and B) as P(A) × P(B)?