Partial derivatives
Why this matters
Every function so far had one input. Real models have many: a line has a slope and an intercept, a network has millions of weights, and the loss depends on all of them at once. So "how does the loss change?" needs a sharper question: how does it change when I nudge this one weight and leave every other one alone? The answer is a partial derivative, written with a curly ∂. It's the ordinary derivative of lessons 72 to 74, applied to one input at a time. By the end of the lesson you'll take partials by hand and by nudging in Python, compute the two slopes that train a straight-line model, and watch PyTorch find the same two with one call.
- Picture a function of two inputs as a table, a contour map and a surface
- Take a partial derivative: change one input, hold the others still, written ∂f/∂x
- Estimate partials in Python by nudging one input at a time, and check them against the hand rule
- Find ∂L/∂w and ∂L/∂b for a line's loss, one slope per weight, and get both from one PyTorch backward()
The math: one input at a time
Purpose. Change when there are several inputs.
A function of two inputs, like f(x, y) = x² + 3y², gives one number for each pair (x, y). You can picture it as a table (rows for x, columns for y), as a contour map (lines joining points of equal height, like a hiking map), or as a surface over a floor.
To ask how f changes with x, hold y still and differentiate as usual, treating y as a constant:
In plain words: ∂f/∂x ("partial f, partial x") is the slope in the x direction with y frozen; ∂f/∂y is the slope in the y direction with x frozen.