Index Ai Academy
Phase 5 · Beyond text

Object detection: boxes, overlap and suppression

Every object and where it is: boxes in two formats, intersection over union, why one object gets many boxes, non-maximum suppression and its threshold, and grading with average precision.

About 45 minutes · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers9

Why it matters

A classifier answers "what is in this picture?" with one label. A self-driving car, a checkout camera or a tool that counts cells under a microscope needs more: every object, and where each one is. That task is object detection, and its answers are boxes. This lesson builds the three ideas every detector rests on, whatever network sits inside it: how to measure whether two boxes agree, how to cut dozens of overlapping guesses down to one per object, and how to grade the result.

What you'll be able to do

  • Describe a detection as a box, a class and a score, and convert between the two ways of writing a box
  • Compute the intersection over union of two boxes, and say why it is harsh on small objects
  • Explain why a detector produces many boxes for one object, and remove the extras with non-maximum suppression
  • Choose a suppression threshold, and say what goes wrong in a crowd
  • Score a detector by matching its boxes to the true ones, and compute average precision
Quick check
A guess before you start. A detector looks at a photo of one dog and returns 30 boxes around it, each shifted a little from the last, each with a confidence score. How should the 30 be turned into one?
  1. Keep whichever of the 30 boxes is the largest
  2. Average the corners of all 30 boxes into a single box
  3. Keep the first box the detector happened to output
  4. Keep the highest-scoring box and drop those that overlap it heavily
Show the answer

Keep the highest-scoring box and drop those that overlap it heavily

The standard method keeps the most confident box, removes every box that overlaps it by more than a set amount, and repeats with what is left. It is called non-maximum suppression. Averaging sounds reasonable and fails as soon as two dogs stand side by side, since their boxes would be averaged into one box between them.

The idea: boxes, overlap and suppression

This rests on Convolutional networks, which gave one label per image, and on Precision, recall and F1, where precision and recall were first counted.

What and where

A detection has three parts: a bounding box, the smallest upright rectangle that holds the object; a class, such as dog; and a score from 0 to 1 saying how confident the detector is.

Coordinates follow the image convention: x runs to the right, y runs down, and (0, 0) is the top left corner. A box is written in one of two ways.

  • Corners: (x1, y1, x2, y2), the top left and bottom right corners. Convenient for measuring overlap.
  • Center and size: (cx, cy, w, h). Convenient for a network to predict, since the center and the size can be adjusted separately.

Every detection library uses one or the other, and mixing them up is the most common bug in detection code.

def to_center(x1, y1, x2, y2):
    return ((x1 + x2) / 2, (y1 + y2) / 2, x2 - x1, y2 - y1)

def to_corners(cx, cy, w, h):
    return (cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2)

box = (4, 6, 14, 12)                           # corners: x from 4 to 14, y from 6 to 12
print("width", box[2] - box[0], "| height", box[3] - box[1], "| area",
      (box[2] - box[0]) * (box[3] - box[1]))
print("as center and size:", to_center(*box))
print("and back to corners:", to_corners(*to_center(*box)))
It runs right here in your browser, with nothing to install.

The box is 10 wide and 6 tall. Its center is at (9, 9), and converting back returns the corners it started from. This lesson uses corners throughout.

Quick check
A network predicts a box as center and size. What does this print: the same box as corners?
cx, cy, w, h = 50, 40, 20, 10
print((cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2))
  1. (40.0, 35.0, 60.0, 45.0)
  2. (50.0, 40.0, 70.0, 50.0)
  3. (30.0, 30.0, 70.0, 50.0)
  4. (40.0, 35.0, 20.0, 10.0)
Show the answer

(40.0, 35.0, 60.0, 45.0)

Half the width, 10, goes to each side of the center's x, and half the height, 5, above and below its y. The box runs from x = 40 to 60 and from y = 35 to 45.

Overlap: intersection over union

To grade a detector, or to notice that two of its boxes are the same object, you need one number for how well two boxes agree. Comparing corners doesn't work: being 5 pixels off is nothing on a bus and everything on a coin. The measure used everywhere is intersection over union, IoU.

IoU(A,B)=area of overlaparea covered by either box\text{IoU}(A, B) = \frac{\text{area of overlap}}{\text{area covered by either box}}

In plain words: the area the two boxes share, divided by the total area they cover together. 1 means the boxes are identical. 0 means they don't touch.

The overlap of two upright boxes is itself a box: it starts at the larger of the two left edges and ends at the smaller of the two right edges, and the same for top and bottom. If that leaves nothing, the overlap is zero. The union is the two areas added, minus the overlap, which would otherwise be counted twice.

def area(b):
    return max(0, b[2] - b[0]) * max(0, b[3] - b[1])

def iou(a, b):
    overlap = area((max(a[0], b[0]), max(a[1], b[1]), min(a[2], b[2]), min(a[3], b[3])))
    return overlap / (area(a) + area(b) - overlap)

truth = (10, 10, 30, 30)                               # the object's true box, 20 by 20
guesses = [("the same box", (10, 10, 30, 30)),
           ("shifted right by a quarter", (15, 10, 35, 30)),
           ("shifted right by half", (20, 10, 40, 30)),
           ("shifted diagonally by half", (20, 20, 40, 40)),
           ("twice the size, same center", (0, 0, 40, 40)),
           ("half the size, same center", (15, 15, 25, 25)),
           ("no overlap", (40, 40, 60, 60))]
for name, guess in guesses:
    print(name.ljust(28), round(iou(truth, guess), 3))

print()
for size in (10, 30, 100, 300):
    a, b = (0, 0, size, size), (3, 3, size + 3, size + 3)
    print("a box", str(size).rjust(3), "pixels wide, 3 pixels off each way | IoU",
          round(iou(a, b), 3))

IoU falls fast. A box shifted by a quarter of its width scores 0.6. Shifted by half, it shares half its area with the truth and scores only 0.333, since the union has grown as the overlap shrank. A box that is the right shape and twice too big scores 0.25, the same as one that is half too small.

By convention a detection counts as correct when its IoU with a true box is at least 0.5. The second table shows what that rule does to small objects. The same 3-pixel error leaves a 100-pixel box at 0.888 and drops a 10-pixel box to 0.325, a miss. Detectors score worse on small objects partly because the measure forgives them less.

Quick check
Two boxes are each 4 wide and 4 tall, and one is shifted 2 to the right of the other. What does this print?
a, b = (0, 0, 4, 4), (2, 0, 6, 4)
overlap = (min(a[2], b[2]) - max(a[0], b[0])) * (min(a[3], b[3]) - max(a[1], b[1]))
union = 16 + 16 - overlap
print(overlap, union, round(overlap / union, 3))
  1. 8 16 0.5
  2. 8 32 0.25
  3. 16 24 0.667
  4. 8 24 0.333
Show the answer

8 24 0.333

The boxes share a strip 2 wide and 4 tall: 8. Together they cover 16 + 16 − 8 = 24, since the shared strip must not be counted twice. 8 / 24 = 0.333. Half of each box overlaps, and the IoU is a third.

Where the boxes come from

A detector doesn't look at an image once and write down a list. It asks the same question at many candidate places, "is an object here, and how sure am I?", and each place answers with a score.

The simplest version is a sliding window: slide a fixed-size box across the image in steps, and score what is under it. The toy detector below does that on a small gray image holding two bright squares. Its score is just the mean brightness under the window. A real detector replaces that with a trained network, and the rest of the procedure is the same.

import numpy as np

rng = np.random.default_rng(0)
# a dim, noisy background: 42 rows, 60 columns
image = rng.normal(0.2, 0.08, size=(42, 60))
objects = [(9, 6, 21, 18), (30, 12, 42, 24)]  # two bright squares, as (x1, y1, x2, y2)
for x1, y1, x2, y2 in objects:
    image[y1:y2, x1:x2] += 0.6

size, step = 12, 3
candidates = []
for y in range(0, image.shape[0] - size + 1, step):
    for x in range(0, image.shape[1] - size + 1, step):
        score = image[y:y + size, x:x + size].mean()
        candidates.append(((x, y, x + size, y + size), round(float(score), 3)))

detections = sorted([c for c in candidates if c[1] > 0.55], key=lambda c: -c[1])
print(len(candidates), "windows scored |", len(detections), "score above 0.55")
for box, score in detections:
    print(box, score)

There are two objects and ten detections. The window that sits exactly on a square scores about 0.8. Its four neighbors, one step to each side, still cover three quarters of the square and score about 0.65, well above anything on the background.

This isn't a flaw of the toy. Every detector behaves this way, because a candidate that is slightly off still sees most of the object. Modern detectors score thousands of candidates per image, and a clear object lights up dozens of them.

Quick check
Why does a detector produce several high-scoring boxes around a single object?
  1. The image is processed several times, once per box
  2. The detector assumes that objects always come in groups
  3. Each box is the detector's guess for a different class
  4. Nearby candidates each see most of the object, and each scores it
Show the answer

Nearby candidates each see most of the object, and each scores it

A candidate box that is a few pixels off still contains nearly all of the object, so it earns nearly the same score. The detector hasn't counted the object many times on purpose. It has answered honestly at many overlapping places, and a later step has to reconcile the answers.

Suppression: one box per object

Non-maximum suppression, NMS, turns each cluster into one box. It is a short greedy loop.

  1. Sort the detections by score, highest first.
  2. Take the best one that is left, and keep it.
  3. Throw away every remaining detection whose IoU with it is above a threshold.
  4. Repeat from step 2 until nothing is left.

The reasoning: the highest score is the best-placed box, and anything overlapping it heavily is the same object seen again.

import numpy as np
import matplotlib.pyplot as plt

def iou(a, b):
    w, h = min(a[2], b[2]) - max(a[0], b[0]), min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    return overlap / (
        (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - overlap)

def nms(detections, threshold):
    kept = []
    for box, score in sorted(detections, key=lambda d: -d[1]):
        # overlaps nothing already kept
        if all(iou(box, k) <= threshold for k, _ in kept):
            kept.append((box, score))
    return kept

rng = np.random.default_rng(0)  # the same image and detector as above
image = rng.normal(0.2, 0.08, size=(42, 60))
for x1, y1, x2, y2 in [(9, 6, 21, 18), (30, 12, 42, 24)]:
    image[y1:y2, x1:x2] += 0.6
detections = []
for y in range(0, 31, 3):
    for x in range(0, 49, 3):
        score = float(image[y:y + 12, x:x + 12].mean())
        if score > 0.55:
            detections.append(((x, y, x + 12, y + 12), round(score, 3)))

for threshold in (0.3, 0.5, 0.7):
    kept = nms(detections, threshold)
    print("threshold", threshold, "|", len(detections), "detections ->", len(kept),
          "kept |", [box for box, score in kept][:3])

fig, axes = plt.subplots(1, 2, figsize=(9, 3.4))
for ax, boxes, title in ((axes[0], detections, "before: 10 detections"),
                         (axes[1], nms(detections, 0.5), "after NMS at 0.5: 2")):
    ax.imshow(image, cmap="gray")
    for (x1, y1, x2, y2), score in boxes:
        ax.add_patch(plt.Rectangle((x1 - 0.5, y1 - 0.5), x2 - x1, y2 - y1, fill=False,
                                   edgecolor="orange", linewidth=1.5))
    ax.set_title(title)
    ax.axis("off")

At a threshold of 0.5, ten detections become two, one on each square. Each neighbor has an IoU of 0.6 with the box it sits beside, which is above 0.5, so it goes. At 0.7 the neighbors survive, since 0.6 is no longer enough overlap to count as the same object, and all ten are kept.

So a high threshold leaves duplicates. A low one has the opposite danger, and it shows up when objects really do overlap.

def iou(a, b):
    w, h = min(a[2], b[2]) - max(a[0], b[0]), min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    return overlap / (
        (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - overlap)

def nms(detections, threshold):
    kept = []
    for box, score in sorted(detections, key=lambda d: -d[1]):
        if all(iou(box, k) <= threshold for k, _ in kept):
            kept.append((box, score))
    return kept

crowd = [                                     # two people standing shoulder to shoulder
    # person 1, seen three times
    ((9, 9, 21, 21), 0.92), ((10, 9, 22, 21), 0.81), ((9, 10, 21, 22), 0.77),
    # person 2, seen three times
    ((15, 9, 27, 21), 0.88), ((14, 9, 26, 21), 0.74), ((14, 10, 26, 22), 0.69),
]
print("IoU of the two people's best boxes:", round(iou(crowd[0][0], crowd[3][0]), 3))
for threshold in (0.3, 0.5, 0.9):
    kept = nms(crowd, threshold)
    print("threshold", threshold, "|", len(kept), "kept |",
          [box for box, score in kept])

The two people's boxes overlap with an IoU of 0.333. At a threshold of 0.5 that is treated as two objects, and NMS returns exactly the two best boxes. At 0.3 it is treated as one, and the second person disappears from the output entirely. At 0.9 all six boxes survive.

No threshold is right for every scene. 0.5 is a common default, and crowds, flocks and shelves of identical products are where it fails. NMS also runs separately for each class, so that a person standing in front of a car doesn't suppress it.

Quick check
What does this print: the boxes that survive suppression at a threshold of 0.5?
def iou(a, b):
    w, h = min(a[2], b[2]) - max(a[0], b[0]), min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    return overlap / (
        (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - overlap)

detections = [("A", (0, 0, 10, 10), 0.6), ("B", (1, 0, 11, 10), 0.9),
              ("C", (30, 0, 40, 10), 0.7), ("D", (2, 0, 12, 10), 0.8)]
kept = []
for name, box, score in sorted(detections, key=lambda d: -d[2]):
    if all(iou(box, k) <= 0.5 for _, k, _ in kept):
        kept.append((name, box, score))
print([name for name, box, score in kept])
  1. ['A', 'C']
  2. ['B']
  3. ['B', 'D', 'C', 'A']
  4. ['B', 'C']
Show the answer

['B', 'C']

B has the highest score and is kept. D overlaps B heavily (IoU 0.818) and is dropped. C is far from B and is kept. A also overlaps B heavily and is dropped. The result is one box for the cluster on the left and one for the object on the right.

Try it now
Worked example

The overlaps behind the picture

The three overlaps the picture used, computed by the lesson's IoU function.

def iou(a, b):
    w = min(a[2], b[2]) - max(a[0], b[0])
    h = min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    area = lambda r: (r[2] - r[0]) * (r[3] - r[1])
    return overlap / (area(a) + area(b) - overlap)

A, B, C, D = (9, 9, 21, 21), (10, 9, 22, 21), (15, 9, 27, 21), (14, 9, 26, 21)
print("A and B:", round(iou(A, B), 2))
print("A and C:", round(iou(A, C), 2))
print("C and D:", round(iou(C, D), 2))
In the lesson you can change this and run it.

A and B share 11 columns of 12 rows, 132 of the 156 square units they cover together: 0.85. A and C share only 6 columns, 72 of 216: 0.33. C and D are a copy of the A and B pair, 0.85. So at 0.5 the neighbors are removed and the two real objects stay.

In your own words
Why is IoU used to compare boxes, and not the distance between their corners?

Write a sentence or two, then compare it with a good answer in the lesson.

In PyTorch: the same ideas on tensors

Scoring a detector

A detector's output is graded against the true boxes, drawn by annotators. Each predicted box has to be declared right or wrong, and the rule has three parts.

  1. Go through the predictions from the highest score down.
  2. Match each to the true box it overlaps most, among those not yet matched.
  3. It is a true positive if that IoU is at least 0.5. Otherwise it is a false positive.

A true box can be matched once. A second prediction on the same object is a false positive, however well placed, which is why leftover duplicates cost points. True boxes that nothing matched are false negatives: missed objects.

From there precision and recall are what they were for classifiers. Walking down the ranked list gives a precision and a recall at every cut-off, and average precision, AP, condenses that list into one number: the mean of the precision at each rank where a true positive appears, counting 0 for every object that was never found.

def iou(a, b):
    w, h = min(a[2], b[2]) - max(a[0], b[0]), min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    return overlap / (
        (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - overlap)

truths = [(8, 6, 20, 18), (30, 10, 42, 22), (50, 30, 58, 38)]
predictions = [((8, 6, 20, 18), 0.95), ((31, 11, 43, 23), 0.90),
               ((2, 26, 14, 38), 0.80), ((29, 9, 41, 21), 0.70),
               ((49, 31, 57, 39), 0.55), ((20, 2, 32, 14), 0.40)]

matched, hits, at_hits = set(), 0, []
for rank, (box, score) in enumerate(sorted(predictions, key=lambda d: -d[1]), start=1):
    free = [i for i in range(len(truths)) if i not in matched]
    best = max(free, key=lambda i: iou(box, truths[i]), default=None)
    overlap = iou(box, truths[best]) if best is not None else 0.0
    hit = overlap >= 0.5
    if hit:
        matched.add(best)
        hits += 1
        at_hits.append(hits / rank)
    print("rank", rank, "| score", score, "| IoU with the best free object",
          str(round(overlap, 2)).ljust(4), "|", "hit " if hit else "miss",
          "| precision", str(round(hits / rank, 2)).ljust(4), "| recall",
          round(hits / len(truths), 2))
print("average precision:", round(sum(at_hits) / len(truths), 3))

Six predictions, three objects. The first two are hits. The fourth is the instructive one: it sits squarely on the second object, which the second prediction has already claimed, so its best free object has an IoU of 0.0 and it is a miss. The fifth finds the last object, and by then precision has slipped to 0.6.

Average precision is (1.0 + 1.0 + 0.6) / 3 = 0.867. It rewards a detector whose confident predictions are right: the same three hits ranked first, second and third would score 1.0.

Benchmarks report mAP, the mean of AP over all the classes, and often average it again over several IoU thresholds from 0.5 to 0.95, which rewards boxes that fit tightly.

Quick check
From the lesson on precision, recall and F1: a detector reports 8 boxes on an image with 10 objects, and 6 of the boxes are correct. What are its precision and recall?
  1. Precision 0.6, recall 0.75
  2. Precision 0.75, recall 0.8
  3. Precision 0.75, recall 0.6
  4. Precision 0.8, recall 0.6
Show the answer

Precision 0.75, recall 0.6

Precision is the share of what was reported that is right: 6 of 8. Recall is the share of what exists that was found: 6 of 10. A detector can trade one for the other by moving the score above which it reports a box.

Quick check
A detector puts two well-placed boxes on the same cat, with scores 0.9 and 0.8. The image holds one cat. How are the two boxes graded?
  1. Both are false positives, since the cat was counted twice
  2. The second replaces the first if its IoU is higher
  3. The first is a true positive and the second a false positive
  4. Both are true positives, since both overlap the cat
Show the answer

The first is a true positive and the second a false positive

Predictions are matched in order of score, and each true box can be matched once. The 0.9 box claims the cat. The 0.8 box then has no free object to match and counts as a false positive. This is how the grading penalizes a detector that leaves duplicates in its output.

Worked example

Worked example: the whole pipeline, and the threshold that decides the score

Run a detector's raw output through suppression and grade it against the truth, at three NMS thresholds. The scene holds four objects, two of them side by side. The raw output has 13 boxes.

def iou(a, b):
    w, h = min(a[2], b[2]) - max(a[0], b[0]), min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    return overlap / (
        (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - overlap)

def nms(detections, threshold):
    kept = []
    for box, score in sorted(detections, key=lambda d: -d[1]):
        if all(iou(box, k) <= threshold for k, _ in kept):
            kept.append((box, score))
    return kept

def grade(predictions, truths):
    matched, hits, at_hits = set(), 0, []
    for rank, (box, score) in enumerate(sorted(predictions, key=lambda d: -d[1]),
                                        start=1):
        free = [i for i in range(len(truths)) if i not in matched]
        best = max(free, key=lambda i: iou(box, truths[i]), default=None)
        if best is not None and iou(box, truths[best]) >= 0.5:
            matched.add(best)
            hits += 1
            at_hits.append(hits / rank)
    return hits, len(predictions) - hits, len(truths) - hits, sum(at_hits) / len(truths)

truths = [(9, 9, 21, 21), (15, 9, 27, 21), (40, 5, 52, 17), (60, 20, 72, 32)]
raw = [((9, 9, 21, 21), 0.92), ((10, 9, 22, 21), 0.81), ((9, 10, 21, 22), 0.77),
       ((15, 9, 27, 21), 0.88), ((14, 9, 26, 21), 0.74), ((14, 10, 26, 22), 0.69),
       ((40, 5, 52, 17), 0.95), ((41, 6, 53, 18), 0.83), ((39, 5, 51, 17), 0.80),
       ((42, 5, 54, 17), 0.62),
       ((61, 21, 73, 33), 0.58), ((60, 22, 72, 34), 0.51), ((30, 28, 42, 40), 0.45)]

print("NMS threshold | boxes kept | right | false alarms | missed | average precision")
for threshold in (0.3, 0.5, 0.9):
    kept = nms(raw, threshold)
    hits, false_alarms, missed, ap = grade(kept, truths)
    print(str(threshold).rjust(13), "|", str(len(kept)).rjust(10), "|",
          str(hits).rjust(5), "|", str(false_alarms).rjust(12), "|",
          str(missed).rjust(6), "|", round(ap, 3))
In the lesson you can change this and run it.

At 0.5, suppression leaves five boxes: one on each of the four objects, and the low-scoring box on empty background, which suppression can't remove since it overlaps nothing. All four objects are found and the false alarm has the lowest score of all, so average precision is 1.0. At 0.3 the two objects standing side by side are merged: four boxes, three of them right, one object missed, and average precision falls to 0.75. At 0.9 nothing is suppressed. All 13 boxes reach the grader, nine count as false alarms, since each object can be matched only once, and average precision is 0.841. It falls less than nine false alarms might suggest, because each duplicate scores below the box it copies and so sits lower in the ranking. The detector is the same on every row. One number in the clean-up step moved its score from 0.75 to 1.0.

Drill

Intersection over union

Write iou(a, b) for two boxes given as corners (x1, y1, x2, y2). Return the area of their overlap divided by the area of their union, rounded to 3 places. Boxes that only touch along an edge don't overlap. If the union has no area, return 0.0.

def iou(a, b):
    return 0.0


print(iou((10, 10, 30, 30), (20, 10, 40, 30)))
In the lesson, your code is checked as you write it.
Drill

Non-maximum suppression

Write nms(detections, threshold). detections is a list of (box, score), with boxes as corners. Go through them from the highest score to the lowest, taking earlier ones first when scores are equal. Keep a detection when its IoU with every detection already kept is at most threshold. Return the kept detections in the order they were kept. iou is already written.

def iou(a, b):
    w, h = min(a[2], b[2]) - max(a[0], b[0]), min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - overlap
    return overlap / union if union > 0 else 0.0


def nms(detections, threshold):
    return detections


crowd = [((9, 9, 21, 21), 0.92), ((10, 9, 22, 21), 0.81), ((15, 9, 27, 21), 0.88),
         ((14, 9, 26, 21), 0.74)]
print(nms(crowd, 0.5))
In the lesson, your code is checked as you write it.
Worked example

All the overlaps at once, in PyTorch

The loop over pairs, as one broadcast: every box against every box, giving a matrix of IoUs.

import torch

boxes = torch.tensor([[9.0, 9, 21, 21], [10, 9, 22, 21],
                      [15, 9, 27, 21], [14, 9, 26, 21]])

def pairwise_iou(a, b):
    lt = torch.maximum(a[:, None, :2], b[None, :, :2])   # (n, m, 2): inner top-left
    rb = torch.minimum(a[:, None, 2:], b[None, :, 2:])   # (n, m, 2): inner bottom-right
    wh = (rb - lt).clamp(min=0)
    inter = wh[..., 0] * wh[..., 1]
    area = lambda x: (x[:, 2] - x[:, 0]) * (x[:, 3] - x[:, 1])
    return inter / (area(a)[:, None] + area(b)[None, :] - inter)

M = pairwise_iou(boxes, boxes)
print(tuple(M.shape))
print([round(float(v), 2) for v in M[0]])
In the lesson you can change this and run it.

Boxes (n, 4) against boxes (m, 4) give a matrix (n, m): the first row is box A against all four, 1.0, 0.85, 0.33 and 0.41. The inner rectangle is the larger top-left against the smaller bottom-right, and clamping at 0 turns a gap into no overlap. This one line replaces the double loop in every detector.

Detection without suppression: set prediction

A transformer detector does not make thousands of candidates and prune them. It starts with a fixed number of learned queries, say 100, and each query cross-attends to the image and outputs one box and one class, including "no object". Training matches the queries to the true boxes one to one, so a query that duplicates another is punished. At inference there is nothing to suppress.

The matching is an assignment problem: pair each true object with the query that fits it best, each query used once. The cell builds the cost as 1 minus the IoU and solves it with SciPy, the same step DETR does with a richer cost.

Worked example

One query per object, matched

Three queries, two true objects. The cost is 1 minus the IoU, and each object gets exactly one query.

import numpy as np
from scipy.optimize import linear_sum_assignment

def iou(a, b):
    w = min(a[2], b[2]) - max(a[0], b[0])
    h = min(a[3], b[3]) - max(a[1], b[1])
    overlap = w * h if w > 0 and h > 0 else 0
    area = lambda r: (r[2] - r[0]) * (r[3] - r[1])
    return overlap / (area(a) + area(b) - overlap)

truths = [(9, 9, 21, 21), (15, 9, 27, 21)]
queries = [(14, 9, 26, 21), (10, 9, 22, 21), (40, 30, 50, 40)]
cost = np.array([[1 - iou(q, t) for t in truths] for q in queries])
print(cost.round(2))
rows, cols = linear_sum_assignment(cost)
for r, c in zip(rows, cols):
    print("query", r, "-> object", c, "IoU", round(1 - cost[r, c], 2))
print("unmatched queries:", sorted(set(range(3)) - set(rows)))
In the lesson you can change this and run it.

Query 0 fits object 1 and query 1 fits object 0, each with an IoU of 0.85; the third query sits on empty ground, costs 1.0 against both, and is left unmatched, so training teaches it to say "no object". No box is removed after the fact: the one-to-one match is what prevents duplicates.

In your own words
What goes wrong when the NMS threshold is too high, and when it is too low?

Write a sentence or two, then compare it with a good answer in the lesson.

In real models: what sits inside a detector

Real detectors replace the brightness score with a trained network, and keep the boxes, the overlap and the grading you built. The tool card shows the library's versions of the two functions you wrote.

The toy detector in this lesson scored windows by brightness. The boxes, the overlap measure, the suppression and the grading are the real ones, and they are the same whichever network produces the scores.

Go deeper What sits inside a real detector

Sliding one window size across an image is far too slow and too rigid. Real detectors keep the idea of candidates and change how they are made.

  • Anchors and grid cells. A convolutional network produces a coarse grid of features, and every cell proposes boxes of a few preset shapes. For each, the network predicts a score, a class and four offsets that nudge the preset shape onto the object. This is called box regression.
  • One stage or two. A two-stage detector, such as Faster R-CNN, first proposes regions and then classifies each. A one-stage detector, such as the YOLO family, predicts boxes and classes in one pass, and is faster.
  • Without NMS. DETR, a transformer-based detector, is trained to output a set with one box per object, matching predictions to true boxes during training, and so doesn't need suppression.

All of them are graded with IoU and average precision, and all but the last finish with NMS.

Where this shows up in an engineer's week

On the job: the bugs are in the boundary. Check the box format first, corners or center and size, and whether coordinates are pixels or fractions of the image. Then pick the suppression threshold from the data, not from a default: a crowd needs a higher one than a parking lot. Grade on a held-out set with the same matching rule you will report, and log the false alarms, since those are what users see.

Goes deeper in

In your own words
A detector scores well on large, separate objects and badly on small or crowded ones. Using this lesson, give one reason for each.

Write a sentence or two, then compare it with a good answer in the lesson.

Practice what you just read

  • 10 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 13 review cards for spaced repetition

Try a quiz question

Two boxes of the same size overlap by exactly half of each box's area. What is their IoU?

  1. One third, since the union is one and a half boxes
  2. One half, since half of each box is shared
  3. Two thirds, since most of the area is shared
  4. One quarter, since each half is counted twice

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 5 · about 36 hours of study

Transformers

Attention, GPT from scratch and pretraining 47 lessons 1 free
  1. Language modeling FreePredict the next token, and score it with cross-entropy.30 min
  2. Tokenization and BPEHow tokenizers are built from data.40 min
  3. A bigram modelThe simplest language model, built from counts.35 min
  4. Sampling strategiesTemperature, top-k and top-p: turning probabilities into text.35 min
  5. Beam search and constrained decodingWhy the likeliest word isn't the likeliest sentence, beam search with log probabilities, its taste for short and repetitive text, and forcing output into a grammar: what that guarantees and what it doesn't.40 min
  6. Neural language modelsWhy counting fails, and what embeddings fix.35 min
  7. Attention: the intuitionEach token decides which other tokens matter to it.30 min
  8. Queries, keys and valuesThe three projections behind attention.35 min
  9. Scaled dot-product attentionThe attention formula, line by line.35 min
  10. Causal maskingStop tokens from seeing the future.25 min
  11. Multi-head attentionSeveral attention patterns in parallel.40 min
  12. Positional encodingsTell the model where each token sits.30 min
  13. The transformer blockAttention, MLP, residuals and norms together.40 min
  14. The GPT architectureA stack of decoder blocks that predicts the next token.40 min
  15. Build a tiny GPTWrite and train a small GPT from scratch.45 min
  16. From your GPT to GPT-2 in Hugging FaceMatch your GPT's tensors to GPT-2's names and shapes, count its 124,439,808 parameters, load renamed weights into your own class, pad a batch and read a config.40 min
  17. Training a GPT wellWarmup and cosine decay, gradient clipping, a validation loss you can trust, saving and loading, and two training mistakes read off the loss curve.35 min
  18. Encoders and embedding modelsBERT-style models and where they fit.40 min
  19. Tokenization: text to integer IDsFollow "The cat sat" into the three integers a model receives.30 min
  20. Embeddings and positional wavesFrom three IDs to the 3 × 8 matrix that enters the first block.35 min
  21. Inside the attention matrixEvery entry of the 3 × 3 attention matrix, computed and steered.40 min
  22. The feed-forward layerExpand to four times d_model and back: shapes, parameters and what the layer stores.35 min
  23. The LM head and samplingFrom the last row to a probability for every token, and the next word.35 min
  24. Pretraining at scaleData, compute and what models learn from the internet.40 min
  25. Scaling lawsHow loss falls predictably with size, data and compute.40 min
  26. Inference and the KV cacheWhy generation is fast after the first token.40 min
  27. The block in today's open models OptionalRMSNorm, SwiGLU, rotary positions and grouped-query attention swapped into the block you built, with the parameter and cache counts before and after.35 min
  28. Long contextEfficient attention and million-token windows.35 min
  29. Mixture of expertsRoute each token to a few specialist sub-networks.35 min
  30. Capstone: diagnose your own GPTEntropy per position, a head's attention pattern and a head ablation on the tiny GPT, then the same three measurements on a GPT you train yourself.45 min
  31. Jacobians: derivatives with shapes OptionalThe derivative of a vector function is a matrix; the chain rule multiplies them, and backprop multiplies by them without building them.35 min
  32. Backprop through a linear layer OptionaldX = dY Wᵀ, dW = Xᵀ dY, db = column sums: derived by shape and by entry, checked on the tiny GPT's own projection.35 min
  33. The softmax Jacobian and p − y OptionalEvery output depends on every input: the full Jacobian, its cheap vector product, and why cross-entropy's gradient is p − y.35 min
  34. Backprop through attention OptionalFive formulas from the output back to queries, keys and values; why masked scores get zero gradient; what FlashAttention recomputes.40 min
  35. Multimodal transformersHow models see images and hear audio.35 min
  36. Object detection: boxes, overlap and suppressionEvery object and where it is: boxes in two formats, intersection over union, why one object gets many boxes, non-maximum suppression and its threshold, and grading with average precision.45 min
  37. Segmentation: a label for every pixelMasks, and how to grade them with IoU and Dice; why a pixel's label needs context and too much context blurs; the shrink-then-enlarge network with skip connections; and turning a mask into countable objects.45 min
  38. Build a vision transformerPatch embeddings, a class token, position embeddings and an unmasked block, trained in the page; then two failures that explain it: no positions means no layout, and few images means losing to a CNN a fraction of its size.45 min
  39. Image-text models in depth: retrieval, negatives and limitsHow to test a shared image-text space with recall at k, what the rest of the batch teaches, what hard and false negatives do, and why a similarity score can't see word order, counts or "no".45 min
  40. Self-supervised learning: labels from the data itselfPretrain an encoder with no labels by matching two views of each image, test it with a linear probe, and see what the augmentations remove, what collapse looks like, and how filling in masked parts differs.45 min
  41. Diffusion 1: turning data into noiseThe easy half of a diffusion model: one step of noise, why the data is shrunk as noise is added, the jump to any step with alpha bar, how to read a noise schedule, and the training examples a denoiser learns from.40 min
  42. Diffusion 2: learning to denoise, and samplingTrain a network to predict noise, then run it a hundred times from pure noise and watch a ring assemble: the sampling loop, why fresh noise goes back in, fewer steps and where they break, and a check that samples are new.45 min
  43. Diffusion 3: images, labels and guidanceFrom points to small images, a label the denoiser can be asked with, and classifier-free guidance: why a model that is told what to draw still often doesn't, how the guidance scale fixes it, and what it breaks when turned up too far.45 min
  44. Diffusion 4: latent diffusion and the whole pipelineWhy image generators diffuse an autoencoder's codes and not pixels: a latent diffusion model in miniature, what the autoencoder must be trained to do, the three parts of a text-to-image system, and where the cost of an image goes.40 min
  45. Sound as numbers: waveforms and spectrogramsA recording as an array of samples, why the sampling rate must be twice the highest pitch, spectrograms and the window trade-off, and the log-mel features a speech model is given.40 min
  46. Recognition and synthesis as a pipelineA whole speech recognizer for a made-up language: features, a guess per frame, collapsing repeats and blanks, a dictionary to repair mistakes, word error rate, and synthesis as the same stages run backward.40 min
  47. The voice agent loop and its latency budgetOne turn of a voice agent, timed: why every stage must stream, how to decide that someone has finished speaking, what to do when they interrupt, and why the slow turns matter more than the average.45 min

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write