Mixed-effects models: letting every group have its own baseline

Cells sit inside fields, fields sit inside wells, wells sit inside experiments — and an ordinary regression has no way to represent that structure. It either pools every cell as if they were independent (understating uncertainty, the pseudoreplication problem) or averages everything down to one number per experiment (throwing away real within-group information). Mixed-effects models solve this by giving every group its own baseline deviation from the overall trend, while still estimating one population-level effect shared across all groups.

Two kinds of effects, one model

$Y_{ij}=X_{ij}\beta+Z_{ij}b_j+\epsilon_{ij}$

• $X_{ij}\beta$ is the fixed effect — the population-average relationship you’d report as “the” result, shared across every group.
• $Z_{ij}b_j$ is the random effect — group $j$‘s own deviation from that population average, treated as drawn from a distribution rather than estimated as a free, unrelated parameter for every group.
• $\epsilon_{ij}$ is leftover noise within group $j$.

Treating $b_j$ as random rather than fixed is what lets the model borrow strength across groups: a well with very little data gets pulled toward the population average rather than reporting a wild, poorly-estimated baseline of its own. This partial pooling is the practical middle ground between “ignore group structure entirely” and “estimate every group in total isolation.”

Worked example: nested wells and experiments

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf

rng = np.random.default_rng(3)
n_experiments, n_wells_per_exp, n_cells_per_well = 4, 3, 15

rows = []
for exp in range(n_experiments):
    experiment_effect = rng.normal(0, 0.6)
    for well in range(n_wells_per_exp):
        well_effect = rng.normal(0, 0.3)
        condition = well % 2  # alternate control/treatment within each experiment
        cell_values = rng.normal(5 + 1.2 * condition + experiment_effect + well_effect, 0.4, n_cells_per_well)
        for v in cell_values:
            rows.append({'experiment': exp, 'well': f'{exp}-{well}', 'condition': condition, 'value': v})

df = pd.DataFrame(rows)
model = smf.mixedlm('value ~ condition', df, groups=df['experiment'], re_formula='~1')
fit = model.fit()
print(fit.summary().tables[1])

groups=df['experiment'] tells the model which level of nesting to treat as the random-effect grouping — every cell within an experiment shares that experiment’s own random deviation, which is exactly what stops the model from treating each cell as independent evidence.

Deciding what’s random

A rule of thumb: anything that would change if the study were repeated — which specific wells, which specific animals, which specific fields of view — is a candidate for a random effect. The condition being tested (control vs. treatment) is usually fixed, because it’s a defined comparison you want to generalise, not a random draw from a population of possible conditions.

Takeaways

• Random effects let each group deviate from the population trend while still estimating one shared population-level effect — a middle ground between full pooling (ignoring group structure) and no pooling (treating every group as unrelated).
• Groups with sparse data get pulled toward the population average automatically (partial pooling), rather than reporting an unstable estimate from too little data.
• Specify groups= at the level that actually generates the dependence in the data (the experiment, well, or animal) — not at the level of the individual measurement.
• Whether something should be fixed or random is a design question: things that would look different in a repeat of the study (specific wells, specific animals) are random; the defined comparison you want to generalise is fixed.

Continue through the concepts

Related notes

Cross-validation without data leakage

Estimate generalisation by keeping preprocessing, tuning, and correlated groups inside correctly separated folds.

Generalized linear models: when your outcome isn't a bell curve

Ordinary linear regression assumes continuous, symmetric, constant-variance noise. GLMs swap in a distribution and link function that actually match counts, binary outcomes, or skewed intensities.

Logistic regression

Model binary-outcome probabilities through log odds and interpret coefficients on odds or probability scales.

Multicollinearity and variance inflation

Understand unstable coefficients caused by strongly correlated predictors and use domain-guided feature reduction.