Index Ai Academy
Foundation · Foundation mathProbability

Bernoulli, binomial, uniform and normal

About 45 minLooking ahead: some prerequisites aren't done yet

Why this matters

A few distributions turn up so often that they have names, and knowing them saves rebuilding a table each time. A single yes-or-no trial is a Bernoulli; the number of yeses in many such trials is binomial; values that are all equally likely are uniform; and sums of many small effects pile up into the bell-shaped normal. Each has a job in ML. Network weights start as small normal numbers; dropout switches each unit off with a Bernoulli coin flip; and the number of test questions a model gets right is binomial, which tells you how much its accuracy would wobble on a different test. This lesson meets all four, simulates them, and puts an error bar on an accuracy.

  • Recognize four distributions: one yes-or-no trial, the count of yeses in n trials, equally likely values, and the bell curve
  • Use the binomial's mean np and variance np(1 − p), the 68–95–99.7 rule, and the z-score
  • Simulate each with rng.random, rng.binomial and rng.normal, stretch and shift a standard normal, and plot a binomial turning into a bell
  • Put an error bar on a model's accuracy, and start a layer's weights, and dropout, with torch.randn, nn.init.normal_ and nn.Dropout
Quick check
Have a guess before you start. A fair coin is tossed 4 times. What is the chance of exactly 2 heads?

The math: four shapes

Purpose. Four shapes that keep appearing.

  • Bernoulli(p): one trial, 1 ("yes") with chance p and 0 with chance 1 − p. A coin, a correct answer, a unit kept by dropout. Mean p.
  • Binomial(n, p): the number of yeses in n independent Bernoulli trials. Its chance of exactly k yeses:
P(k)=C(n,k) pk(1−p)n−k,mean np,variance np(1−p)P(k) = C(n, k)\, p^k (1-p)^{n-k}, \qquad \text{mean } np, \quad \text{variance } np(1-p)

In plain words: count the ways to place k yeses (lesson 87), times the chance of any one such sequence.

For 4 coin tosses: 0.0625, 0.25, 0.375, 0.25, 0.0625 for 0 to 4 heads, mean 4 × 0.5 = 2.