Statistical note

The t-test: comparing means with uncertainty

A practical guide to one-sample, paired, and independent t-tests, their assumptions, and responsible interpretation.

10 August 20267 min read

The t-test asks whether an observed difference in means is large relative to the uncertainty in that difference. It does not ask whether two samples merely look different, and it does not measure scientific importance by itself.

Two sampling distributions compared by a t-test

Three common forms

A one-sample t-test compares one sample mean with a reference value.
An independent t-test compares means from two unrelated groups.
A paired t-test analyses within-pair differences, such as measurements before and after treatment on the same cells or specimens.

For two independent groups, the core signal-to-noise idea is

$t=\frac{\bar{x}_1-\bar{x}_2}{SE(\bar{x}_1-\bar{x}_2)}.$

Welch’s t-test is usually the safer independent-samples default because it does not assume equal variances.

import numpy as np
from scipy import stats

control = np.array([10.2, 9.8, 10.7, 11.0, 10.4])
treated = np.array([11.5, 12.1, 11.8, 12.4, 11.7])
result = stats.ttest_ind(treated, control, equal_var=False)
print(result.statistic, result.pvalue, result.confidence_interval())

Assumptions worth checking

Observations should be independent at the level used in the analysis. This is especially important in microscopy: thousands of cells from one dish are not thousands of independent biological replicates. The paired test assumes that the differences are approximately normal. With small samples, severe skew and outliers can strongly affect the result.

Report more than a p-value

Report group summaries, the estimated mean difference, a confidence interval, an effect size, the exact test used, degrees of freedom, and the p-value. A small p-value can coexist with a biologically trivial effect when the sample is large.

A practical decision

Use a t-test when the scientific question is genuinely about a mean, the study design matches the selected form, and the sampling unit is defensible. If the estimand is a median, rank shift, or a broader distributional difference, a non-parametric or model-based analysis may answer the question more directly.

Case study: TIRF intensity before and after correction

If the same fields are measured before and after background correction, use scipy.stats.ttest_rel; pairing removes field-to-field baseline variation. If treated and control wells are unrelated, use ttest_ind(..., equal_var=False). In both cases, the independent unit is the well or experiment, not every detected particle pooled together.