Statistical note

Classification metrics from the confusion matrix

Accuracy, balanced accuracy, precision, recall, specificity, F1, MCC, and choosing metrics under class imbalance.

24 June 202611 min read

Binary predictions produce true positives $TP$ , false positives $FP$ , true negatives $TN$ , and false negatives $FN$ .

$\mathrm{precision}=\frac{TP}{TP+FP},\qquad \mathrm{recall}=\frac{TP}{TP+FN},$

$\mathrm{specificity}=\frac{TN}{TN+FP},\qquad F_1=\frac{2PR}{P+R}.$

Confusion matrix and derived metrics

from sklearn.metrics import (
    confusion_matrix, classification_report, balanced_accuracy_score,
    matthews_corrcoef,
)

y_true = [0, 0, 0, 0, 1, 1, 1, 1]
y_pred = [0, 0, 0, 1, 0, 1, 1, 1]
print(confusion_matrix(y_true, y_pred))
print(classification_report(y_true, y_pred, digits=3))
print(balanced_accuracy_score(y_true, y_pred))
print(matthews_corrcoef(y_true, y_pred))

Case study: rare mitosis detection

Accuracy can remain high when a model misses most rare events. Recall measures captured mitoses; precision measures how many alerts are real. Balanced accuracy averages class recalls. The Matthews correlation coefficient summarizes all four cells and remains informative under imbalance.

Always define the positive class, averaging rule (micro, macro, or weighted), decision threshold, and unit of evaluation.

Functions: sklearn.metrics.confusion_matrix, precision_recall_fscore_support, balanced_accuracy_score, matthews_corrcoef, and classification_report.

Case study: rare mitosis detection

Related notes