Index Ai Academy
Phase 4 · Deep learningNeural networks

The artificial neuron

About 25 minLooking ahead: some prerequisites aren't done yet

Why this matters

Every neural network is built from one small unit, copied over and over. That's true of a spam filter. It's also true of a chatbot with hundreds of billions of parameters (adjustable numbers). The unit is called a neuron. You've already trained one without calling it that: logistic regression, from Phase 3. This lesson shows exactly what one neuron can do, and the kind of pattern it can never learn. That gap is why networks need layers, which is the story of this phase.

  • Compute a neuron's output: a weighted sum plus a bias, passed through an activation
  • Recognize logistic regression as a single neuron
  • See a neuron's decision boundary as a straight line
  • Explain why one neuron can't learn XOR, and why that matters

A neuron is a tiny scorecard

Picture deciding whether to go for a run after work. You weigh a few things. Sunshine pushes you toward yes. Rain pushes you toward no, and pushes hard. You also have a starting mood. Some days you're keen before you even look outside, and some days you're not.

A neuron makes a decision the same way. It takes in a few numbers, called inputs. It gives back one number, its output. It gets there in two steps.

Step 1: the weighted sum. Each input has a weight. A weight says how much its input matters, and in which direction. A positive weight pushes the output up. A negative weight pushes it down. The neuron multiplies each input by its weight and adds up the results. Then it adds one more number, the bias. The bias is the starting mood: where the score sits before any input has a say.

Call the result zz. Write the first input as x1x_1. Its weight is w1w_1. The second input is x2x_2. Its weight is w2w_2, and so on. The letter bb is the bias. Say there are nn inputs. Then the weighted sum is z=w1x1+w2x2+⋯+wnxn+b=w⋅x+bz = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b = \mathbf{w} \cdot \mathbf{x} + b. The three dots mean "and so on, for every input".

In plain words: multiply each input by its weight, add them all up, then add the bias.

The short form on the right uses bold letters for whole lists. The bold w\mathbf{w} is the list of weights. The bold x\mathbf{x} is the list of inputs. The dot between them is the dot product from Phase 2: multiply in pairs, then add. In NumPy that's np.dot(w, x).

Try it with small numbers. Two inputs are x1=1x_1 = 1 and x2=3x_2 = 3. Their weights are w1=2w_1 = 2 and w2=−0.5w_2 = -0.5. The bias is b=−1b = -1. Then z=2×1+(−0.5)×3−1=2−1.5−1=−0.5z = 2 \times 1 + (-0.5) \times 3 - 1 = 2 - 1.5 - 1 = -0.5.