Index Ai Academy
Foundation · Foundation mathCalculus

Limits: what a value heads toward

About 37 minLooking ahead: some prerequisites aren't done yet

Why this matters

Lesson 70 measured change across a gap. The derivative, next lesson, asks for change at a single point, where the gap is zero, and dividing by zero makes no sense. Limits are the way around it: instead of asking what happens at the point, ask what the values head toward as you get closer and closer. That one idea makes the derivative precise. It also gives words for what can go wrong with a function: a hole, a corner, a jump. Those words matter in ML, because activation functions are judged by them, and because code that gets too close to dividing by zero needs a guard.

  • Read a limit from a table, approaching a point from both sides
  • See that a limit can exist where the formula itself breaks, at a hole
  • Tell smooth points, corners and jumps apart, and say which have a limit
  • Judge activation functions by the same test, in plain Python and with PyTorch's F.relu and torch.sigmoid
  • Add a tiny epsilon so code never divides by zero, as real training code does with torch.sqrt(v + eps)
Quick check
Have a guess before you start. What do the numbers 1.9, 1.99, 1.999, 1.9999 head toward?

The math: getting closer without arriving

Purpose. Make sense of "as the gap shrinks to nothing".

Take f(x) = (x² − 1)/(x − 1). At x = 1 the formula gives 0 ÷ 0: it breaks. But try values close to 1: f(0.9) = 1.9, f(0.99) = 1.99, f(1.01) = 2.01. From both sides the values close in on 2. We say the limit of f as x approaches 1 is 2:

lim⁡x→1x2−1x−1=2\lim_{x \to 1} \frac{x^2 - 1}{x - 1} = 2

In plain words: as x gets as close to 1 as you like, from either side, f(x) gets as close to 2 as you like.

The point itself doesn't matter, only the approach. Here there's a hole at x = 1: the graph is the line y = x + 1 with one point missing.