Index Ai Academy
Foundation · Foundation mathProbability

Probability: from counts to chances

About 43 minLooking ahead: some prerequisites aren't done yet

Why this matters

A probability is a number for how likely something is, from 0 (impossible) to 1 (certain). For simple experiments, like rolling a die, it comes straight from counting: the ways the thing can happen, divided by all the ways anything can happen. When counting is hard, you can run the experiment many times and see how often it happens. Both ideas sit at the center of machine learning: a language model's entire output is a list of probabilities, one for each possible next word, that add up to 1. This lesson builds probability from counts, checks it by simulation, and turns a camp log into the kind of next-word table a model produces.

  • Find a probability as a share: the ways an event can happen over all the equally likely ways
  • Use the rules every probability obeys: between 0 and 1, all outcomes adding to 1, and P(not A) = 1 − P(A)
  • Estimate a chance by simulation with a seeded random generator, the Monte Carlo method, and watch more trials settle
  • Turn word counts into a next-word distribution, the whole output of a language model, in Python and with PyTorch's counts / counts.sum()
Quick check
Have a guess before you start. A fair die is rolled. What is the chance of an even number?

The math: a share of the possibilities

Purpose. A number for how likely.

An experiment has outcomes, like the six faces of a die. An event is a set of outcomes, like "even". When the outcomes are equally likely:

P(A)=ways A can happenall waysP(A) = \frac{\text{ways } A \text{ can happen}}{\text{all ways}}

In plain words: the share of the possibilities that count as A.

P(even) = 3/6 = 0.5. A bag of 10 cards with 4 red: P(red) = 4/10 = 0.4. Every probability is between 0 and 1, and the probabilities of all the outcomes add up to 1: something has to happen.