The artificial neuron

A neuron is a tiny scorecard

Picture deciding whether to go for a run after work. You weigh a few things. Sunshine pushes you toward yes. Rain pushes you toward no, and pushes hard. You also have a starting mood. Some days you're keen before you even look outside, and some days you're not.

A neuron makes a decision the same way. It takes in a few numbers, called inputs. It gives back one number, its output. It gets there in two steps.

Step 1: the weighted sum. Each input has a weight. A weight says how much its input matters, and in which direction. A positive weight pushes the output up. A negative weight pushes it down. The neuron multiplies each input by its weight and adds up the results. Then it adds one more number, the bias. The bias is the starting mood: where the score sits before any input has a say.

Call the result $z$. Write the first input as $x_1$. Its weight is $w_1$. The second input is $x_2$. Its weight is $w_2$, and so on. The letter $b$ is the bias. Say there are $n$ inputs. Then the weighted sum is $z = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b = \mathbf{w} \cdot \mathbf{x} + b$. The three dots mean "and so on, for every input".

In plain words: multiply each input by its weight, add them all up, then add the bias.

The short form on the right uses bold letters for whole lists. The bold $\mathbf{w}$ is the list of weights. The bold $\mathbf{x}$ is the list of inputs. The dot between them is the dot product from Phase 2: multiply in pairs, then add. In NumPy that's np.dot(w, x).

Try it with small numbers. Two inputs are $x_1 = 1$ and $x_2 = 3$. Their weights are $w_1 = 2$ and $w_2 = -0.5$. The bias is $b = -1$. Then $z = 2 \times 1 + (-0.5) \times 3 - 1 = 2 - 1.5 - 1 = -0.5$.

print(1 * 3 + 2 * -1 + 0.5)

Step 2: the activation. The neuron passes $z$ through one more function, called the activation function. Call that function $f$. The output is $a = f(z)$, and $a$ is called the activation. Put both steps together:

In plain words: take the weighted sum, add the bias, then pass the result through the activation function. What comes out is the neuron's answer.

In this lesson $f$ is the sigmoid you met in logistic regression. It squashes any number into the range 0 to 1. A big positive $z$ gives an answer close to 1. A big negative $z$ gives an answer close to 0. A $z$ of exactly 0 gives 0.5. For the numbers above, $z = -0.5$ gives $a \approx 0.378$, about a 38% chance.

With the sigmoid as $f$, a neuron is exactly logistic regression from Phase 3. So a neuron with a sigmoid outputs a probability. The weights decide how much each input matters. The bias shifts the point where the neuron "switches on".

import math

x = [1, 3]        # opened the last email (1 = yes), days since last visit
w = [2.0, -0.5]   # weights: how much each input matters
b = -1.0          # bias: the starting point

z = sum(wi * xi for wi, xi in zip(w, x)) + b   # weighted sum plus bias
a = 1 / (1 + math.exp(-z))                      # sigmoid activation
print(z, round(a, 3))

w = [1, 2]
x = [4, -1]
b = -1
z = w[0] * x[0] + w[1] * x[1] + b
print(z)

The same neuron in NumPy

Here's a neuron that guesses whether a customer will churn, meaning stop buying from you. It has two inputs. The first is days since their last order, in hundreds (so 1.5 means 150 days). The second is how many orders they place per year.

The weight on days is 1.8. It's positive, so a long gap pushes the churn chance up. The weight on orders is −0.6. It's negative, so frequent buying pushes the chance down. The bias is −1.

import numpy as np

def neuron(x, w, b):
    z = np.dot(w, x) + b
    return 1 / (1 + np.exp(-z))          # sigmoid activation

# A "will this customer churn?" neuron: inputs are days since last order (in 100s) and orders per year
w = np.array([1.8, -0.6])
b = -1.0
for x in [[0.2, 9], [1.5, 2], [3.0, 1]]:
    print(f"inputs {x}  →  output {neuron(np.array(x), w, b):.3f}")

Follow the first customer by hand. They ordered 20 days ago and buy 9 times a year. The weighted sum is 1.8 × 0.2 = 0.36, plus −0.6 × 9 = −5.4. Add the bias of −1, and $z$ is −6.04. That's a big negative number, so the sigmoid gives 0.002: almost no chance of churning.

The second customer has $z = 0.5$ and an output of 0.622. The third hasn't ordered for 300 days and buys once a year. Their output is 0.978, so the neuron is nearly sure they'll leave.

The name "neuron" comes from a loose comparison with brain cells. A brain cell adds up incoming signals and fires once they pass a threshold. The comparison stops there. An artificial neuron is nothing more than the formula above.

A neuron draws a straight line

The sigmoid gives more than 0.5 exactly when $z$ is above 0. So every possible input lands on one of two sides. Where $z > 0$, the neuron says "yes". Where $z < 0$, it says "no". The dividing line, where $z$ is exactly 0, is called the decision boundary: $\mathbf{w} \cdot \mathbf{x} + b = 0$.

In plain words: the boundary is every input where the weighted sum plus the bias comes to exactly zero. On one side the neuron says yes, and on the other it says no.

With two inputs, that boundary is always a straight line. With three inputs, it's a flat sheet, a plane. With more inputs, it's the same kind of flat cut in more directions, called a hyperplane. So a single neuron can split two groups only if a straight line (or flat cut) can split them. Groups that can be split this way are called linearly separable.

Here's a small example. Take weights $w_1 = 1$ and $w_2 = 1$, and bias $b = -1.5$. Then $z = x_1 + x_2 - 1.5$. Feed in each corner of a square, where both inputs are 0 or 1. Only $(1, 1)$ makes $z$ positive: $1 + 1 - 1.5 = 0.5$. The line $x_1 + x_2 = 1.5$ cuts that one corner off from the other three.

w1, w2, b = 1.0, 1.0, -1.5   # a hand-picked neuron
for x1, x2 in [(0, 0), (0, 1), (1, 0), (1, 1)]:
    z = w1 * x1 + w2 * x2 + b
    print((x1, x2), z, "yes" if z > 0 else "no")

It prints no for the first three corners and yes for $(1, 1)$.

Three tiny logic problems

Now let a neuron learn its own line. Each problem below has the same four inputs: (0,0), (0,1), (1,0) and (1,1). Each one wants a different answer, 0 or 1, for each input:

• AND gives 1 only when both inputs are 1.
• OR gives 1 when at least one input is 1.
• XOR, short for "exclusive or", gives 1 when exactly one input is 1. A hallway light with a switch at each end can be wired this way.

The code trains one neuron per problem with gradient descent. The update lines are the same kind you used for logistic regression. In each plot, red dots want 1 and blue dots want 0. The gray line is the boundary the neuron learned.

import numpy as np
import matplotlib.pyplot as plt

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=float)
problems = {"AND": [0, 0, 0, 1], "OR": [0, 1, 1, 1], "XOR": [0, 1, 1, 0]}

def train(X, y, lr=1.0, steps=5000):
    w, b = np.zeros(2), 0.0
    for _ in range(steps):
        p = 1 / (1 + np.exp(-(X @ w + b)))
        w -= lr * X.T @ (p - y) / len(y)
        b -= lr * np.mean(p - y)
    return w, b

fig, axes = plt.subplots(1, 3, figsize=(10, 3.3))
for ax, (name, y) in zip(axes, problems.items()):
    y = np.array(y, dtype=float)
    w, b = train(X, y)
    pred = (1 / (1 + np.exp(-(X @ w + b))) > 0.5).astype(int)
    print(f"{name:<4} weights {np.round(w, 1)}  bias {b:5.1f}  predictions {pred}  correct {int((pred == y).sum())}/4")
    ax.scatter(X[:, 0], X[:, 1], c=y, cmap="coolwarm", s=120, edgecolors="gray")
    if abs(w[1]) > 1e-6:
        xs = np.linspace(-0.5, 1.5, 2)
        ax.plot(xs, -(w[0] * xs + b) / w[1], color="gray")
    ax.set_xlim(-0.5, 1.5)
    ax.set_ylim(-0.5, 1.5)
    ax.set_title(name)

AND and OR come out perfect, 4 out of 4. One straight line puts the 1s on one side and the 0s on the other. The AND neuron learns a weight of about 10.7 on each input and a bias of about −16.2. That's close to the hand-picked line $x_1 + x_2 = 1.5$, with bigger numbers. Bigger numbers push the outputs closer to 0 and 1.

XOR fails. Its 1s sit on opposite corners, (0,1) and (1,0). Its 0s sit on the other two corners. No straight line can put both 1s on one side and both 0s on the other. The neuron's weights and bias stay at 0, so it outputs 0.5 for every input. That counts as a 0 every time, so it gets only 2 of the 4 right. Its plot has no line at all.

Why XOR mattered

In 1969, Marvin Minsky and Seymour Papert published a book called Perceptrons. It showed that single-layer networks can't represent patterns like XOR. The book helped cause years of reduced interest in neural networks.

The fix was known in principle: stack neurons in layers. Then later neurons can combine the lines drawn by earlier ones. But training stacked layers needed a fast way to work out every weight's gradient: how much the error changes when that weight changes. Backpropagation, made popular in 1986, provided it. The next few lessons build exactly that.

From one neuron to an LLM

Roughly speaking, a large language model is a huge number of these units arranged in layers. Each one takes a weighted sum of its inputs and applies an activation. The model isn't capable because each unit is smarter. It's capable because of how many units there are, how they're wired, and the training that sets billions of weights.

For marketers: a lead score that adds up weighted signals and squashes the result to 0–1 is a single neuron. It works when the signals add up independently. Sometimes the answer depends on a combination. Say a big company is a good lead when its buying team is small, but a poor lead when the team is large. For a small company, it's the other way around. That flip is the XOR pattern, and no weighted sum can capture it. Then you need more than one neuron.