Index Ai Academy
Foundation, before Phase 0 · about 96 hours of study

Foundation math

From whole numbers to the math inside a neural network: each idea by hand, in Python, and in AI. 118 lessons, about 96 hours of study, in nine tracks. All of them are free to try without an account.

The model map

Where the math lives.

One diagram of a model learning, the same in every lesson. Each lesson lights the parts its math is used in; here every part links to the math behind it and the lessons that teach it.

The math behind each part
What you'll be able to read

A real training loop, line by line.

These eight lines are how most of today's models learn. By the end you'll know the math behind every one of them, and you'll have written the same lines yourself.

  1. model = nn.Sequential(nn.Linear(25, 8), nn.ReLU(), nn.Linear(8, 3))Lesson 110: A layer and a small network as matrices
  2. opt = torch.optim.Adam(model.parameters(), lr=1e-3)Lesson 80: Faster descent: momentum and Adam
  3. for x, y in loader:Lesson 64: Tensors, shapes and broadcasting
  4.     logits = model(x)Lesson 56: Matrix multiplication
  5.     loss = F.cross_entropy(logits, y)Lesson 105: Surprise, entropy and cross-entropy
  6.     opt.zero_grad()Lesson 82: Automatic differentiation: how PyTorch finds every slope
  7.     loss.backward()Lesson 74: The product rule and the chain rule
  8.     opt.step()Lesson 79: Gradient descent
What you'll learn

Nine tracks, in order.

118 lessons, about 96 hours of study. Open a lesson to read it, or start at the first.

Numbers · 13 lessons, about 9.5 hours

  1. Numbers and place value FreeWhat a digit's place is worth, the number line, and Python as a calculator that never tires.36 min
  2. Multiplying and dividing: groups FreeEqual groups, quotient and remainder, and cutting training data into batches.35 min
  3. Order of operations, and names for numbers FreeBrackets, powers, then multiply and divide, then add and subtract; variables hold numbers.36 min
  4. Numbers below zero FreeThe number line through zero, sign rules, distance from zero, and an error's direction and size.37 min
  5. Fractions: equal parts, equal fractions, adding FreePieces of a whole, equal fractions, and re-cutting to add; accuracy as a fraction.45 min
  6. Multiplying and dividing fractions FreeA share of a share, dividing by flipping, and why chances in a row get small.35 min
  7. Decimals and rounding FreeFractions in place value, rounding, and why 0.1 + 0.2 isn't exactly 0.3.36 min
  8. Percentages FreePer hundred, percent change, percentage points, and the accuracy and error rates of models.35 min
  9. Ratios, rates and proportion FreeSharing in a ratio, rates per unit, scaling, and splitting data 80/10/10.35 min
  10. Powers and roots FreeRepeated multiplying, the laws of exponents, square roots, and squared error.43 min
  11. Very big and very small numbers FreePowers of ten, e-notation like 3e-4, orders of magnitude, and a model's size in memory.36 min
  12. Bits, bytes and floats: how numbers are stored FreeBase 2, what n bits can hold, a float as sign, power and digits, and a tensor's memory.35 min
  13. Review: find the slip (numbers) FreeBroken calculations to fix, mixed practice from the whole module, and its formula sheet.34 min

Algebra · 16 lessons, about 13 hours

  1. Letters for numbers FreeVariables, terms and like terms, substitution, and a model's formula as a Python function.41 min
  2. Expanding and factoring FreeBrackets as areas, pulling out common factors, and a squared error expanded into a bowl.41 min
  3. Solving equations: keep the balance FreeThe same move on both sides, undoing steps in reverse, and running a model's formula backward.41 min
  4. Rearranging a formula FreeMaking any letter the subject, round-trip checks, and planning a training run.36 min
  5. Inequalities and true-or-false FreeRanges, flipping the sign, if and elif, and the thresholds, clips and ReLU inside a model.42 min
  6. Functions: a rule from input to output Freef(x) notation, tables and graphs, domain and range, loops over lists, and loss as a function.42 min
  7. Chaining and undoing functions Freeg(f(x)), inverses that undo, scaling and unscaling, and a network as a chain.42 min
  8. Straight lines: slope and intercept FreeRise over run, the line through two points, and a model's weight and bias as slope and intercept.42 min
  9. Two equations, two unknowns FreeSubstitution and elimination, where lines cross, and why models are fitted, not solved.42 min
  10. Quadratics: bowls and their lowest point FreeRoots by factoring and by formula, the discriminant, the vertex, and the best weight of a model.42 min
  11. Exponentials: growth and decay FreeThe same factor each step, the number e, and decaying learning rates.43 min
  12. Logarithms: the power question FreeTo what power?, the laws of logs, and adding log-probabilities instead of multiplying.45 min
  13. Sequences and Σ FreeArithmetic and geometric sequences, Σ as a loop that adds, and mean squared error.40 min
  14. Weighted and moving averages FreeWeights that add to 1, a window average, the exponential moving average, and smoothing a loss curve.35 min
  15. Sets, and reading the symbols FreeSets, union and intersection, ℝⁿ and f : ℝⁿ → ℝᵐ, and reading a paper's first equation.41 min
  16. Review: find the slip (algebra) FreeBroken algebra and code to fix, mixed practice from the whole module, and its formula sheet.34 min

Shapes, angles and waves · 5 lessons, about 3.5 hours

  1. Coordinates and distance FreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
  2. Angles, degrees and radians FreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
  3. Sine and cosine FreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
  4. Waves: height, speed and shift FreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
  5. Review: find the slip (shapes and waves) FreeBroken geometry and code to fix, mixed practice from the module, and its formula sheet.29 min

Vectors and matrices · 21 lessons, about 17 hours

  1. Vectors: lists of numbers that point FreeA list, a point and an arrow; components, dimension, NumPy arrays, and data as vectors.42 min
  2. Adding and scaling vectors FreeComponent-wise sums, scalar multiples, mixes, and the weight update w − ηg.43 min
  3. Length and distance: norms FreeL2, L1 and max norms, unit vectors, distance, and the norms inside losses and clipping.42 min
  4. The dot product FreeMultiply in pairs and add; a·a as squared length; a @ b; one neuron as a dot product.43 min
  5. Angle, cosine similarity and projection FreeDirection without length, perpendicular vectors, shadows, and search by meaning.45 min
  6. Perpendicular directions: bases and coordinates FreeOrthonormal directions, coordinates as dot products, rebuilding a vector, and taking a direction out.35 min
  7. Matrices: tables of numbers FreeRows, columns and entries; two-dimensional arrays and slices; a data set as a matrix.41 min
  8. Adding, scaling and transposing matrices FreeEntry-by-entry sums and scalings, the transpose, mean images, and tables turned around.36 min
  9. A matrix times a vector FreeEach row dotted with the vector, the size rule, A @ x, and a layer acting on one example.42 min
  10. Matrix multiplication FreeRow meets column, the size rule, order matters, and a batch through a layer: X W + b.45 min
  11. Identity, diagonal, triangular and symmetric FreeFour patterns and their jobs: rescaling, distance tables, and the causal mask.37 min
  12. Matrices as movements of space FreeStretches, flips, shears and rotations; columns as landing spots; chaining; a layer moving data.42 min
  13. Determinant and inverse FreeThe area factor ad − bc, singular matrices, undoing a matrix, and redundant features.42 min
  14. Solving systems by elimination FreeAugmented matrices, row moves, back-substitution, the three outcomes, and fitting weights.45 min
  15. Independence, span and rank FreeRedundant vectors, span, rank, outer products, and low-rank updates like LoRA.42 min
  16. Eigenvectors: directions that only stretch FreeAv = λv, checking candidates, np.linalg.eig, and the main direction of a data cloud.41 min
  17. SVD and PCA: the important directions first FreeA matrix as pieces by size, the best rank-k copy, compression, and PCA maps of data.45 min
  18. Tensors, shapes and broadcasting FreeAxes and shapes, reshape, axis=, broadcasting, and shapes through a small model.43 min
  19. One-hot vectors and lookup tables: embeddings FreeA one-hot row picks a table's row, a sentence looked up as vectors, and a score for every word.35 min
  20. Sliding dot products: convolution FreeA small pattern slid along a list and over a grid, output sizes, stride and padding, and shared weights.40 min
  21. Review: find the slip (vectors and matrices) FreeShape slips to fix, mixed practice from the module, and its formula sheet with shapes.36 min

Calculus · 16 lessons, about 13.5 hours

  1. Rates of change FreeChange over change, secants, rates from readings with np.diff, and reading a loss curve.41 min
  2. Limits: what a value heads toward FreeApproaching from both sides, holes, corners and jumps, epsilon, and judging activations.37 min
  3. The derivative: slope at a point FreeShrinking secants, f′(x) and dy/dx, Δy ≈ f′Δx, and which way a weight should move.45 min
  4. Derivative rules FreeConstants, powers, multiples, sums, eˣ and ln x, checked numerically, and a squared error's slope.41 min
  5. The product rule and the chain rule FreeProducts and chains of functions, forward then backward, and one neuron's dL/dw.45 min
  6. Lowest points and curvature Freef′ = 0, the second-derivative test, tables, local and global minima, and convex losses.43 min
  7. Standing in for a curve near a point FreeTangent lines, a curvature term, growing error, and why learning rates must be small.37 min
  8. Partial derivatives FreeOne input at a time, ∂, nudging in Python, and the two slopes of a line's loss.41 min
  9. The gradient FreeAll the slopes as a vector, steepest climb and descent, a numerical gradient, and the loss gradient.42 min
  10. Gradient descent FreeStepping against the gradient, the learning rate, a training loop, and fitting a line.45 min
  11. Faster descent: momentum and Adam FreeMomentum and Adam by hand, the bias correction, and three optimizers down one narrow valley.40 min
  12. The chain rule with many paths: backpropagation by hand FreeGraphs, multiply along paths and add across them, gradient checks, and a 2-2-1 network by hand.45 min
  13. Automatic differentiation: how PyTorch finds every slope FreeA tape of steps walked backward, and requires_grad, backward(), .grad, zero_grad() and no_grad().40 min
  14. Jacobians and Hessians FreeSlopes of vector functions, a layer's Jacobian, second partials, and bowls, domes and saddles.42 min
  15. Integrals: adding up thin slices FreeStrip and trapezoid sums, summing undoes differencing, probability as area, and AUC.43 min
  16. Review: find the slip (calculus) FreeCalculus slips to fix, mixed practice from the module, and its formula sheet.36 min

Probability · 9 lessons, about 7.5 hours

  1. Counting: choices, orders and groups FreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
  2. Probability: from counts to chances FreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
  3. Combining events: and, or, independent FreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
  4. Conditional probability and Bayes' rule FreeConditionals from tables, why P(A | B) isn't P(B | A), Bayes' rule, and precision and recall.45 min
  5. Random variables and distributions FreeUncertain numbers, the probability of a range, sampling by slices, and sampling versus argmax.42 min
  6. Expected value, variance and standard deviation FreeProbability-weighted averages, spread in the data's own units, and a training loss as an expected value.42 min
  7. Bernoulli, binomial, uniform and normal FreeYes-or-no trials, counts of yeses, the 68-95-99.7 rule, z-scores, and an accuracy's error bar.45 min
  8. Two variables at once: covariance and correlation FreeJoint tables and marginals, covariance, correlation, the covariance matrix, and features that carry the same information.45 min
  9. Review: find the slip (probability) FreeProbability slips to fix, mixed practice from the module, and its formula sheet.36 min

Statistics and information · 11 lessons, about 9 hours

  1. Describing data: center, spread and scale FreeMean, median and mode, variance and standard deviation, percentiles, outliers, and standardizing features.42 min
  2. Samples, the law of large numbers and the central limit theorem FreeSamples and populations, the standard error σ/√n, the bell of averages, and error bars on test scores.43 min
  3. Is the difference real? Comparing two models FreePaired differences, a bootstrap interval, what a p-value is and isn't, and why many tries make lucky wins.40 min
  4. Maximum likelihood FreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
  5. Linear regression: the best line FreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
  6. Logistic regression: a line that answers yes or no FreeThe sigmoid, the log loss, its slope p − y, a training loop, and choosing a threshold.45 min
  7. Scoring a classifier: confusion table, precision, recall, F1 FreeThe four cells, accuracy, precision, recall and F1, and what moving a threshold does to each.35 min
  8. Softmax: scores into probabilities FreeExponentiate and share out, shifting and temperature, the safe softmax, and a classifier's last step.43 min
  9. Surprise, entropy and cross-entropy FreeSurprise in bits, entropy, cross-entropy and KL divergence, perplexity, and a language model's loss.45 min
  10. KL divergence: how far one distribution is from another FreeKL by hand, cross-entropy as entropy plus KL, why it isn't symmetric, and a soft-target loss.35 min
  11. Review: find the slip (statistics and information) FreeStatistics slips to fix, mixed practice from the module, and its formula sheet.38 min

Putting it together · 8 lessons, about 7 hours

  1. One neuron, start to finish FreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
  2. A layer and a small network as matrices FreeTwo layers on a batch, the shape of every matrix, adding a hidden unit, and XOR solved.45 min
  3. Training a small network FreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
  4. The same network in PyTorch Freenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
  5. Attention as matrix math FreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
  6. Residual connections and layer norm FreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
  7. A transformer block, start to finish FreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
  8. Capstone: a sign reader from the formulas FreeBuild, train and test a trail-sign classifier with NumPy alone, and report its accuracy with an error bar.45 min

Transposition of formulas · 19 lessons, about 15 hours

  1. Transposing step by step: undo in reverse order FreeA formula's layers, the opposite of each, peeling from the outside in, conditions, and round-trip checks.35 min
  2. Formulas with fractions: clear them first FreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
  3. When the letter appears twice: collect and factor FreeGathering the letter on one side, factoring it out, dividing by the bracket, and rearranging precision and F1.35 min
  4. Squares, roots and powers FreeSquare roots with ±, squaring to undo a root, cube and nth roots, and the √d inside attention.40 min
  5. When the letter is squared and plain: quadratics FreeRearranging into ax² + bx + c = 0, the formula, completing the square, and choosing the root that fits.35 min
  6. Letters in the power: exponentials and logarithms FreeTaking logs to bring a power down, exponentiating to free a letter, and the logit that undoes the sigmoid.40 min
  7. Rearranging inequalities: when the sign flips FreeThe four moves on an inequality, negatives and reciprocals that flip it, logs below 1, and sample sizes.35 min
  8. Inverse functions, lines and scaling FreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
  9. Sequences and sums: find n, a, d and r FreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
  10. Combining formulas: substitute and eliminate FreePutting one formula inside another to remove a letter, and sizing a training run from a time or money budget.35 min
  11. Review: rearrange anything (algebra) FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
  12. Angles, triangles and waves: rearranging with sin, cos and tan FreeSides and angles from SOH CAH TOA, principal values, arcs, polar coordinates, and a wave's time and period.45 min
  13. Vector equations: rearranging with arrows FreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
  14. Matrix equations: multiply by the inverse, on the correct side FreeNo dividing by a matrix: inverses from the left or right, order, transposes, and the normal equations.40 min
  15. Rearranging in calculus: slopes, steps and constants FreeSetting a slope to zero, Newton's step, the chain rule turned around, constants from a point, and learning rates.45 min
  16. Probability formulas: solve for the missing chance FreeConditionals, Bayes' rule for the prior, the or rule, expected values, variance, and z-scores turned around.45 min
  17. Statistics and information: n, z, logits and bits FreeSample sizes from error bars, inputs from probabilities, logit gaps from softmax, and chances from bits and losses.40 min
  18. Planning a model: rearranging the formulas of a working network FreeLayer widths from parameter counts, sizes from memory and compute, the attention scale, and a training plan.40 min
  19. Final test: transpose any formula FreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Where it fits

The parts of the system this course works on.

Each part, what it does, the real tool for it and the lessons that teach it. In a lesson, the same map lights the part you are working on.

Where this sits in the model

A model learning: forward from data to a loss, backward to an update, and what sits around training.

Before and after

Where it starts, and what you earn.

Before you start

Nothing. This phase starts from your very first line of Python, and every lesson in it is free to try. Already know some? The placement test marks off what you know, so you only study what's new to you.

When you finish

This phase leads to:

Finish every lesson in this phase and you earn a certificate of completion, "Foundation math (Foundation)", with its own link you can share. Pass the phase's checkpoint, 12 questions the academy's server draws and marks itself, and it becomes a verified certificate; the checkpoint is also how lessons you skipped in the placement test count.

The full course

All thirteen phases.

Every lesson on one page, or the AI engineer roadmap, which shows what you can skip.

  1. Tools
  2. Building with LLMs
  3. Math for ML engineers
  4. Machine learning
  5. Deep learning
  6. Transformers
  7. Modern LLM engineering
  8. Software engineering
  9. APIs
  10. Model Context Protocol
  11. Agent harnesses
  12. Professional practice
  13. Foundation math

How a lesson works

Read a little, run real Python in your browser, change it, and answer a short quiz. Exercises check your code and hint at what went wrong. What you learn comes back as review cards, spaced out over the weeks. See what you'll build.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write.

Start the Foundation

Start with Numbers and place value.

What a digit's place is worth, the number line, and Python as a calculator that never tires.