Index Ai Academy
Foundation · Foundation mathProbability

Random variables and distributions

About 42 minLooking ahead: some prerequisites aren't done yet

Why this matters

Lesson 88 found the chance of an event. A random variable goes further: it's an uncertain number, like a die's face or tomorrow's rentals, described completely by its distribution, the chance of every value it can take. Once you have the distribution, you can answer any question about the number, and you can draw samples from it: pick values at random, with the likely ones picked more often. That last idea is how a language model writes. At every step it has a distribution over the next word, and generating text means sampling from it, over and over. This lesson builds distributions, samples them by hand and in Python, and writes a few words that way.

  • Describe an uncertain number as a random variable with a distribution: a table or bars that sum to 1
  • Find the probability of a range of values, and read a curve's area as probability for continuous values
  • Sample from a distribution with rng.choice, count the samples into a histogram, and watch it approach the bars
  • Generate text by sampling next words, compare it with always taking the most likely one, and reshape the odds with temperature and top-k in PyTorch
Quick check
Have a guess before you start. A spinner lands on 1 half the time, 2 a third of the time and 3 otherwise. What is P(3)?

The math: an uncertain number and its distribution

Purpose. A full description of an uncertain number.

A random variable X is a number whose value depends on chance: the face of a die, the number of heads in ten tosses, the next word's position in a vocabulary. Its distribution gives the probability of every value:

x1234
P(X = x)0.50.30.150.05

In plain words: a table (or a bar chart) of each value and its chance. The chances are between 0 and 1 and add up to 1.

Questions about X become sums: P(X ≥ 3) = 0.15 + 0.05 = 0.2; P(X is even) = 0.3 + 0.05 = 0.35.