Statistical note

Regression metrics: MAE, RMSE, R², and deviance

Choosing a loss that matches error costs and outcome distributions, from continuous measurements to count predictions.

12 June 202611 min read

For residuals $e_i=y_i-\hat y_i$ ,

$MAE=\frac1n\sum_i|e_i|,\qquad RMSE=\sqrt{\frac1n\sum_i e_i^2}.$

RMSE penalizes large errors more strongly. $R^2=1-\sum e_i^2/\sum(y_i-\bar y)^2$ compares squared error with a mean-prediction baseline and can be negative out of sample.

Absolute, squared, and count-aware regression errors

from sklearn.metrics import (
    mean_absolute_error, root_mean_squared_error, r2_score,
    mean_poisson_deviance,
)

y_true = [3, 5, 9, 12, 20]
y_pred = [4, 4, 8, 15, 18]
print(mean_absolute_error(y_true, y_pred))
print(root_mean_squared_error(y_true, y_pred))
print(r2_score(y_true, y_pred))
print(mean_poisson_deviance(y_true, y_pred))

Case study: density-map counting

For bacteriophage counts, Poisson deviance respects count-like mean–variance structure better than arbitrary percentage error. MAE communicates average count error. Evaluate bias and error versus count magnitude; a single aggregate score can hide systematic undercounting in dense images.

MAPE is unstable near zero. Report multiple complementary metrics and uncertainty across independent test experiments.

Functions: sklearn.metrics.mean_absolute_error, root_mean_squared_error, r2_score, mean_poisson_deviance, and mean_pinball_loss.

Case study: density-map counting

Related notes