Index Ai Academy
Glossary · Foundation · Foundation math

Automatic differentiation

Also written: autodiff, reverse-mode differentiation

Software that records each step of a calculation, then walks back through the record to find the slope of the result with respect to every input.

Try it now

The tape ends empty, as PyTorch's does after backward(). To walk it again, you run the forward pass again, which records a fresh tape.

Learn it properly

Automatic differentiation: how PyTorch finds every slope · A tape of steps walked backward, and requires_grad, backward(), .grad, zero_grad() and no_grad(). About 40 minutes.

Keep learning

More in plain English.

1286 terms in the glossary, each with the lesson that teaches it.

1286 terms

The glossary

Every word you'll meet in AI and machine learning, explained in everyday language.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write