Statistical note

Comparing more than two groups

ANOVA, Welch ANOVA, Kruskal–Wallis, and Friedman tests with assumptions, effect decomposition, and post-hoc cautions.

6 July 202610 min read

An omnibus test asks whether a set of groups is compatible with a shared model. Classical one-way ANOVA uses

$F=\frac{MS_{\mathrm{between}}}{MS_{\mathrm{within}}},$

so large between-group variation relative to within-group variation challenges equal means.

Between-group and within-group variation

from scipy import stats

control = [10, 11, 9, 12, 10]
dose_1 = [12, 13, 11, 14, 13]
dose_2 = [16, 14, 17, 15, 18]

print(stats.f_oneway(control, dose_1, dose_2))
print(stats.kruskal(control, dose_1, dose_2))

# Repeated measurements on the same experimental units
print(stats.friedmanchisquare(control, dose_1, dose_2))

Choosing the family

Classical ANOVA assumes independent errors and equal group variances. Welch’s formulation relaxes equal variance. Kruskal–Wallis compares rank distributions for independent groups. Friedman handles blocked or repeated samples.

Case study: three microscopy treatments

Do not follow a significant omnibus result with every pairwise test uncorrected. Predefine contrasts or apply multiplicity control. Report group estimates and intervals; the omnibus p-value does not identify which groups differ or whether differences matter biologically.

Functions: scipy.stats.f_oneway, kruskal, friedmanchisquare, tukey_hsd, and dunnett.

Choosing the family

Case study: three microscopy treatments

Related notes