Statistical note

Cramér–von Mises goodness-of-fit test

Cramér–von Mises goodness-of-fit test: theory, interpretation, Python computation, and a scientific-imaging case study.

30 April 20266 min read

A test statistic orders datasets by incompatibility with a null model. Its reference distribution converts the observed statistic into a p-value.

Concept figure for Cramér–von Mises goodness-of-fit test

Mathematical definition

The central quantity is

$W^2=integral (F_n-F_0)^2 dF_0.$

This expression states what is being counted, averaged, ranked, or compared. Its scale and direction must be interpreted in the context of the data and sampling design.

Compute it in Python

import numpy as np
from scipy import stats
x = np.random.default_rng(3).normal(size=25)
print(stats.cramervonmises(x, 'norm'))

The function is scipy.stats.cramervonmises. Inspect its current signature and return object in the official documentation for the version installed in your environment.

Interpretation and cautions

The design determines the test: independent, paired, repeated, categorical, or distributional questions are not interchangeable. Report an effect estimate and interval alongside the test.

A numerical value is not self-interpreting. Compare it with a baseline, uncertainty interval, operational threshold, or competing model, and retain the underlying observations or confusion counts.

Scientific-imaging case study

For integrated distribution discrepancy in imaging data, preserve the biological sampling unit and avoid treating correlated cells or pixels as independent replicates.

Split train, validation, and test data at the specimen or experimental level. Report variability across independent repeats so that the metric describes generalization rather than leakage.

Mathematical definition

Compute it in Python

Interpretation and cautions

Scientific-imaging case study

Related notes