Probability: from counts to chances
Why this matters
A probability is a number for how likely something is, from 0 (impossible) to 1 (certain). For simple experiments, like rolling a die, it comes straight from counting: the ways the thing can happen, divided by all the ways anything can happen. When counting is hard, you can run the experiment many times and see how often it happens. Both ideas sit at the center of machine learning: a language model's entire output is a list of probabilities, one for each possible next word, that add up to 1. This lesson builds probability from counts, checks it by simulation, and turns a camp log into the kind of next-word table a model produces.
- Find a probability as a share: the ways an event can happen over all the equally likely ways
- Use the rules every probability obeys: between 0 and 1, all outcomes adding to 1, and P(not A) = 1 − P(A)
- Estimate a chance by simulation with a seeded random generator, the Monte Carlo method, and watch more trials settle
- Turn word counts into a next-word distribution, the whole output of a language model, in Python and with PyTorch's counts / counts.sum()
The math: a share of the possibilities
Purpose. A number for how likely.
An experiment has outcomes, like the six faces of a die. An event is a set of outcomes, like "even". When the outcomes are equally likely:
In plain words: the share of the possibilities that count as A.
P(even) = 3/6 = 0.5. A bag of 10 cards with 4 red: P(red) = 4/10 = 0.4. Every probability is between 0 and 1, and the probabilities of all the outcomes add up to 1: something has to happen.