Segmentation metrics: IoU, Dice, and boundary error

For predicted pixels $P$ and reference pixels $G$,

$IoU=\frac{|P\cap G|}{|P\cup G|},\qquad Dice=\frac{2|P\cap G|}{|P|+|G|}.$

For a binary mask, Dice equals the pixel-level F1 score, and $Dice=2IoU/(1+IoU)$.

import numpy as np
from sklearn.metrics import jaccard_score, f1_score

truth = np.array([[0, 1, 1], [0, 1, 0], [0, 0, 0]]).ravel()
pred  = np.array([[0, 1, 1], [0, 1, 1], [0, 0, 0]]).ravel()
print("IoU", jaccard_score(truth, pred))
print("Dice", f1_score(truth, pred))

Case study: cell segmentation

Pixel overlap can look excellent even when adjacent cells are merged, because most foreground pixels remain correct. Add object-level matching, split/merge counts, boundary distance, per-image distributions, and downstream measurement error. Define how empty masks and small objects are handled.

Macro averaging gives each class equal weight; micro averaging pools pixels and can be dominated by background. scikit-learn supplies overlap primitives, while object and boundary metrics often require domain-specific code.

Functions: sklearn.metrics.jaccard_score, f1_score, confusion_matrix, and multilabel_confusion_matrix.

Continue through the concepts

Related notes

Cellpose: segmentation that generalises because it doesn't assume a shape

Why predicting a flow field toward each object's centre, instead of a mask directly, let one trained model segment cell types it had never seen — without retraining.

Van Valen et al.: CNNs learn live-cell tracking from ~100 annotated cells

One of the earliest demonstrations that a small amount of manual annotation was enough to train a CNN that out-tracked classical live-cell segmentation methods — the work that grew into DeepCell.

Mask R-CNN: instance segmentation as detection plus one branch

How adding a single mask-prediction branch onto an existing object detector turned 'find the objects' into 'find and precisely outline every object' — and the alignment fix that made small-object masks accurate.

SAM 2: keeping a mask's identity across a video

How a streaming-memory architecture let promptable segmentation extend from single images to video, tracking an object's identity through occlusion and motion in real time.