Index Ai Academy
Foundation · Foundation mathVectors and matrices

Solving systems by elimination

About 45 minLooking ahead: some prerequisites aren't done yet

Why this matters

Lesson 22 solved two equations in two unknowns by eliminating one letter. Real problems have more: three unknowns, ten, a thousand. Elimination scales up by writing only the numbers, in a matrix, and following one method: clear the numbers below the diagonal, row by row, until the matrix is a staircase, then read the answer off from the bottom up. It's the method behind np.linalg.solve, it shows plainly when a system has no answer or endless ones, and it answers a question at the heart of ML: which weights make a model fit these examples exactly, and what to do when no weights can.

  • Write a system of equations as an augmented matrix [A | b]
  • Use row moves to reach a staircase, then substitute back up to the answer
  • Tell the three outcomes apart: one solution, none, or infinitely many
  • Solve for a model's weights exactly, in NumPy and PyTorch, and see why more examples than weights needs a best fit instead
Quick check
Have a guess before you start. x + y = 5 and x − y = 1. What are x and y?

The math: row moves down to a staircase

Purpose. Many equations, many unknowns, one method.

Write each equation as a row of its coefficients, with the right-hand side after a bar. The system

x + y + z = 6, 2x + 3y + z = 11, x + 3y + 4z = 19

becomes the augmented matrix:

(11162311113419)\left(\begin{array}{ccc|c} 1 & 1 & 1 & 6 \\ 2 & 3 & 1 & 11 \\ 1 & 3 & 4 & 19 \end{array}\right)

In plain words: each row is one equation; each column before the bar is one unknown.

Three row moves keep the solution the same: swap two rows; multiply a row by a nonzero number; add a multiple of one row to another. Each is something you could do to the equations themselves without changing what solves them.