Bernoulli, binomial, uniform and normal
Why this matters
A few distributions turn up so often that they have names, and knowing them saves rebuilding a table each time. A single yes-or-no trial is a Bernoulli; the number of yeses in many such trials is binomial; values that are all equally likely are uniform; and sums of many small effects pile up into the bell-shaped normal. Each has a job in ML. Network weights start as small normal numbers; dropout switches each unit off with a Bernoulli coin flip; and the number of test questions a model gets right is binomial, which tells you how much its accuracy would wobble on a different test. This lesson meets all four, simulates them, and puts an error bar on an accuracy.
- Recognize four distributions: one yes-or-no trial, the count of yeses in n trials, equally likely values, and the bell curve
- Use the binomial's mean np and variance np(1 − p), the 68–95–99.7 rule, and the z-score
- Simulate each with rng.random, rng.binomial and rng.normal, stretch and shift a standard normal, and plot a binomial turning into a bell
- Put an error bar on a model's accuracy, and start a layer's weights, and dropout, with torch.randn, nn.init.normal_ and nn.Dropout
The math: four shapes
Purpose. Four shapes that keep appearing.
- Bernoulli(p): one trial, 1 ("yes") with chance p and 0 with chance 1 − p. A coin, a correct answer, a unit kept by dropout. Mean p.
- Binomial(n, p): the number of yeses in n independent Bernoulli trials. Its chance of exactly k yeses:
In plain words: count the ways to place k yeses (lesson 87), times the chance of any one such sequence.
For 4 coin tosses: 0.0625, 0.25, 0.375, 0.25, 0.0625 for 0 to 4 heads, mean 4 × 0.5 = 2.