Logistic regression: a line that answers yes or no
Why this matters
Linear regression predicts an amount. Many questions want a yes or no instead: is this email spam, is this sign an arrow, will this hiker need a rescue. A model for those should answer with a chance between 0 and 1, and a straight line can't: it runs off below 0 and above 1. Logistic regression keeps the line as a score and squeezes the score through the sigmoid, an S-shaped curve that turns any number into a chance. It is trained by maximum likelihood (lesson 100), and its loss, the log loss, has a slope so simple, p − y, that the same expression returns in nearly every classifier and language model. A single neuron with a sigmoid is exactly this model, so this lesson is also the first complete neuron of the section: score, squeeze, loss, slope, step.
- Turn a line's score into a chance with the sigmoid, σ(z) = 1/(1 + e⁻ᶻ), and read it off the S-curve
- Measure a yes-or-no prediction with the log loss, and see why it punishes confident mistakes
- Derive and use the loss's slope with respect to the score, p − y, in a short training loop
- Train a two-feature sign detector, read it as one neuron, choose a threshold, and get the same loss and slopes from PyTorch's nn.BCEWithLogitsLoss
The math: score, squeeze, loss
Purpose. Predict the chance of a yes, and measure how good that prediction was.
The score. As in linear regression, each example gets a score z = w·x + b: a weighted sum of its features plus a bias. A high score means "probably yes", a low one "probably no", but it can be any number.
The squeeze. The sigmoid turns the score into a chance:
| z | −4 | −2 | 0 | 2 | 4 |
|---|---|---|---|---|---|
| σ(z) | 0.018 | 0.119 | 0.5 | 0.881 | 0.982 |
A score of 0 gives 0.5, a coin toss. Big positive scores approach 1 and big negative ones approach 0, never reaching either. The decision boundary is where z = 0: on one side the model leans yes, on the other no.