Object Detection

A detector looks at an image and proposes many candidate boxes, each with a class and a confidence score. Raw output is messy: the same dog might get ten slightly shifted boxes. Two simple ideas clean it up.

Intersection over union (IoU) measures how much two boxes overlap: the area they share divided by the area they cover together. 1 means identical, 0 means no overlap.

Non-max suppression (NMS) removes duplicates:

1. Drop boxes below a confidence threshold.
2. Take the most confident remaining box and keep it.
3. Delete every other box of the same class whose IoU with it is above a threshold (often 0.5).
4. Repeat with the next most confident box.

Detectors come in two styles. Two-stage detectors (the R-CNN family) first propose regions, then classify each one: accurate but slower. One-stage detectors like YOLO predict all boxes in a single pass of a CNN, fast enough for video.

The same IoU measure is also used to score detectors: a prediction counts as correct only if its IoU with a true box is high enough.

The math

For boxes $A$ and $B$:

$$\text{IoU}(A, B) = \frac{|A \cap B|}{|A \cup B|} = \frac{|A \cap B|}{|A| + |B| - |A \cap B|}$$