Maximum likelihood

Work with the log instead

With 2,000 visitors, the likelihood is a product of 2,000 numbers below 1. You saw the problem in the lesson on exponentials and logarithms. The answer gets so small that the computer rounds it to zero. That's called underflow.

The fix is to take the log. The log turns multiplying into adding, so nothing shrinks to zero. And the log always goes up when its input goes up. So the peak stays at the same $p$.

The curly letter $\ell$ (read "ell") stands for the log-likelihood. Here "log" means the natural log, np.log or math.log in code.

In plain words: add up $\log p$ once for each buyer. Add $\log(1 - p)$ once for each visitor who didn't buy. The bigger the total, the better $p$ explains the data. The total is always negative, so "bigger" means closer to zero.

Back to the tiny shop, with 3 buyers, 1 visitor who left, and $p = 0.75$:

• 3 × log 0.75 ≈ 3 × (−0.288) ≈ −0.863
• 1 × log 0.25 ≈ −1.386
• total ≈ −2.249

That's the log of the likelihood you found before, 0.105. Same information, written as a sum.

import math

p = 0.75
log_lik = 3 * math.log(p) + 1 * math.log(1 - p)
print(round(log_lik, 4))
print(round(math.log(p * p * p * (1 - p)), 4))

Both lines print −2.2493. Now the full shop with 2,000 visitors. The code below tries 600 rates between 0.01 and 0.07. It works out the log-likelihood of each one and keeps the best. (np.argmax gives the position of the largest value in a list.)

import numpy as np
import matplotlib.pyplot as plt

conversions, visitors = 64, 2000
p = np.linspace(0.01, 0.07, 600)
log_lik = conversions * np.log(p) + (visitors - conversions) * np.log(1 - p)

best = p[np.argmax(log_lik)]
plt.plot(p, log_lik)
plt.axvline(best, linestyle="--", color="gray")
plt.xlabel("conversion rate p")
plt.ylabel("log-likelihood")
print(f"MLE by search: {best:.4f}   conversions / visitors: {conversions / visitors:.4f}")

The curve is a hill. Its top sits at $64 / 2000 = 0.032$, which is conversions ÷ visitors. The same holds for any rate: the MLE is successes divided by trials. Your everyday guess was maximum likelihood all along.

import math

clicks, sent = 3, 20     # 3 people clicked out of 20 emails sent

for p in [0.05, 0.10, 0.15, 0.20, 0.30]:
    log_lik = clicks * math.log(p) + (sent - clicks) * math.log(1 - p)
    print(p, round(log_lik, 3))

The same trick for other models

Every model below has its own setting to estimate. In each case, maximum likelihood gives an answer you may already have guessed. A small "hat" on a letter, as in $\hat{p}$ (read "p-hat"), means "the estimate worked out from the data".

Here's each row in plain words:

• Bernoulli is one yes-or-no try per visitor, like the shop. The best rate is the share of yeses.
• Poisson models counts in a fixed window, like orders per hour. Its setting is $\lambda$ (lambda), the average rate. The best estimate is the plain average of your counts.
• Normal is the bell curve. Its middle, $\mu$ (mu), is estimated by the mean. Its spread, the variance $\sigma^2$ (sigma squared), is estimated by the average squared distance from the mean. Maximum likelihood divides by $n$, the number of values, not by $n - 1$.
• Linear regression draws the best straight line through a cloud of points. You'll meet it in Phase 3. "Least squares" means choosing the line that makes the squared misses as small as possible.

print(30 / 200)

That last row explains why least squares is the default for drawing a line. If the noise around the line follows a bell curve, the least-squares line is the maximum-likelihood line.

Training a language model is maximum likelihood

A language model reads text one token at a time. A token is a word or a piece of a word. At each position, the model gives a probability to every possible next token. Its parameters are all the numbers inside it that training adjusts. Together, they're written $\theta$ (theta).

Think back to the shop. There, you multiplied one probability per visitor. Here, you multiply one probability per token: the probability the model gave to the token that actually came next. That product is the likelihood of the training text. Training looks for the $\theta$ that makes it as large as possible. In logs, the goal is:

Take it one piece at a time:

• $P_\theta(\ldots)$ is the probability the model, with parameters $\theta$, gives to the token at position $t$. The bar $\mid$ reads "given", as in the conditional probability lesson.
• $\sum_t$ (a capital sigma) means "add up over every position $t$".
• $\max_\theta$ means "find the parameters that make this total as large as possible".

In plain words: at every position in the training text, take the log of the probability the model gave to the token that really came next. Add them all up. Training adjusts the model to make that total as big as it can.

Now flip the sign and take the average instead of the total. You get the average negative log-likelihood, which is the training loss. The cross-entropy lesson, coming up soon, shows this is the same as cross-entropy loss. Gradient descent pushes the loss down. That's the same as pushing the likelihood up.

A small example. Say a model gives the real next tokens probabilities 0.5 and 0.25:

• log-likelihood: log 0.5 + log 0.25 ≈ −0.693 + (−1.386) ≈ −2.079
• average loss: flip the sign and divide by 2 tokens, ≈ 1.040

The code does the same for a five-token sentence and two models.

import numpy as np

# Probabilities a model gave to the tokens that actually came next in one sentence
probs_good = np.array([0.6, 0.8, 0.4, 0.7, 0.9])
probs_bad = np.array([0.1, 0.3, 0.05, 0.2, 0.4])
for name, p in [("good model", probs_good), ("weak model", probs_bad)]:
    print(f"{name}: log-likelihood {np.log(p).sum():.2f}, average loss {-np.log(p).mean():.3f}")

The good model scores a log-likelihood of −2.11 and an average loss of 0.422. The weak model scores −9.03 and 1.806. Higher likelihood, lower loss: they're the same statement made two ways.