Index Ai Academy
Foundation · Foundation mathAlgebra

Exponentials: growth and decay

About 43 minLooking ahead: some prerequisites aren't done yet

Why this matters

Some things grow by adding the same amount each step; others grow by multiplying by the same factor. The second kind, exponential growth, starts slowly and then outruns any straight line. Its mirror image, decay, shrinks by a factor each step. Both are everywhere in machine learning: learning rates decay over training, and the exponential function turns a model's raw scores into positive numbers it can compare. You'll also rebuild the number e from scratch, with a loop that pays interest more and more often, and see why exponential growth always overtakes a straight line in the end.

  • Recognize exponential growth and decay: the same factor each step
  • Compare exponential growth with a straight line, and see where e = 2.718… comes from
  • Compute exponentials in Python with ** and math.exp, and with a loop that multiplies each step
  • Build a decaying learning-rate schedule, and see how e^score makes any score positive
Quick check
Have a guess before you start. Which is bigger after 10 steps: starting at 1 and doubling each step, or starting at 1 and adding 100 each step?

The math: the same factor each step

Purpose. Describe things that multiply by the same amount each step.

Linear growth adds the same amount each step: 100, 150, 200, 250. Exponential growth multiplies by the same factor each step: 100, 200, 400, 800. In symbols:

y=a⋅rty = a \cdot r^{t}

In plain words: start at a, and multiply by r once for each of t steps. With r = 2 it doubles every step.

When r is above 1 the value grows; when r is between 0 and 1 it shrinks, which is decay. Halving each step, r = 0.5, takes 80 to 40, 20, 10. It never quite reaches zero: there's always half of something left.