Index Ai Academy
Foundation · Foundation mathPutting it together

One neuron, start to finish

About 45 minLooking ahead: some prerequisites aren't done yet

Why this matters

This module adds no new math. Every formula in it was taught and practiced in Modules 1 to 7; what's new is running them together, the way a real system does. It starts with the smallest thing that learns: one neuron. Its training step is five familiar pieces, a weighted sum, an activation, a loss, slopes by the chain rule and an update, and once they run as one loop, the neuron learns. You'll take that step by hand, in NumPy and in PyTorch, then see exactly what one neuron can and cannot learn, which explains why networks stack them: the next lesson.

  • Follow one neuron through a full training step by hand: weighted sum, activation, loss, slopes, update
  • Name the earlier lesson behind each of the five steps
  • Write the five steps as five lines of NumPy, then loop them until the loss reaches a target
  • Take the same step in PyTorch with backward() and optim.SGD, and get the hand numbers back
  • See what a single neuron can learn, a split of the plane by a straight line, and what it can't
Quick check
Have a guess before you start. A neuron gets inputs (2, 1), weights (0.5, −1) and bias 0.5. What is its weighted sum z = w·x + b?

The math: five steps, all known

Purpose. Run one complete training step of one neuron by hand.

A neuron with two inputs has two weights w₁ and w₂ and a bias b. Here the inputs are x = (2, 1), the weights (0.5, −1), the bias 0.5, and the target, the answer it should give, is y = 1. One training step:

  1. Weighted sum (lesson 50): z = w₁x₁ + w₂x₂ + b = 1 − 1 + 0.5 = 0.5.
  2. Activation (lesson 71): a = relu(z) = max(0, z) = 0.5.
  3. Loss (lesson 79): L = (a − y)² = (0.5 − 1)² = 0.25.
  4. Slopes (lessons 74 and 81): backward through each step, multiplying local slopes.
  5. Update (lesson 79): every weight moves against its slope, by the learning rate.

Steps 1 to 3 are the forward pass, step 4 the backward pass. Step 4 is the only one with any work in it, and it's lesson 81's graph again: seven small steps, walked backward.