Index Ai Academy
Foundation · Foundation mathCalculus

Lowest points and curvature

About 43 minLooking ahead: some prerequisites aren't done yet

Why this matters

Training a model means finding the weights that make the loss as small as possible. So the question "where is this function lowest?" is the goal of nearly everything in machine learning. Calculus gives a sharp answer for smooth functions: at a lowest point the slope is zero, because the curve has stopped falling and not yet started rising. The second derivative, the slope of the slope, then says whether that flat spot is the bottom of a bowl or the top of a dome. This lesson finds turning points by hand and in Python, and then meets the problem real models face: a bumpy loss with more than one valley, where the bottom you find depends on where you start.

  • Find turning points by setting the slope to zero
  • Use the second derivative to tell a bowl (a lowest point) from a dome (a highest point)
  • Find a minimum in Python by solving f′ = 0, or by searching a table of values, and plot it to check the ends
  • Tell local from global minima, and see why a convex loss is easy and a bumpy one isn't
  • Read PyTorch's x.grad as the slope, 0 at a turning point, and watch optim.SGD end in different valleys from different starts
Quick check
Have a guess before you start. You're walking along a valley floor in fog. How can you tell you're at the very bottom?

The math: where the slope is zero

Purpose. Find where a function is smallest, or largest.

At a lowest or highest point of a smooth curve, the tangent is flat: f′(x) = 0. These are turning points. For f(x) = x² − 4x + 5:

f′(x)=2x−4=0⇒x=2,f(2)=1f'(x) = 2x - 4 = 0 \quad\Rightarrow\quad x = 2, \quad f(2) = 1

In plain words: solve "slope = 0" for x, then put that x back in to get the height.

To the left of 2 the slope is negative (falling); to the right, positive (rising). So x = 2 is a lowest point, a minimum, with value 1.