Statistical note

Limits of detection and quantification

A reference-level guide to limits of detection and quantification: theory, assumptions, diagnostics, worked bioimaging analysis, exercises, and reporting.

12 August 202614 min read

Levelintermediate

Prerequisites

After this note, you can

define limits of detection and quantification precisely
check its identifying assumptions and diagnostics
apply it to hierarchical biological-image data without pseudoreplication

This chapter belongs to Reliability and measurement. It is designed as a working reference: begin with the scientific estimand, establish what the design identifies, then choose computation and diagnostics. Software output is the final step, not the definition of the question.

Concept map for Limits of detection and quantification

1. When this concept answers the question

Use Limits of detection and quantification when the data structure and scientific target require it. First state the population, observational unit, intervention or exposure, outcome, comparison, and time horizon. In bioimaging, distinguish pixels, objects, fields, wells, specimens, patients, acquisition sessions, and instruments. The level receiving treatment or independently sampled usually controls the effective replication.

Do not select the method solely because a variable has a familiar data type. The same count can be a Poisson outcome, an overdispersed count, a zero-inflated mixture, a repeated measure, or a spatial point pattern depending on how it arose.

2. Formal framework

The central mathematical object is

$LOD=\mu_{blank}+k\sigma_{blank}.$

Define every symbol before calculation. Separate the estimand (the population quantity), estimator (the rule applied to data), estimate (its realised value), and uncertainty statement. Frequentist probability concerns repeated procedures conditional on fixed parameters; Bayesian probability conditions on observed data and a specified model.

3. Assumptions checklist

Design: Were units sampled or assigned in a way that supports the target claim?
Independence: Which observations share a specimen, well, field, time series, batch, or annotator?
Measurement: Are calibration, segmentation error, detection limits, and missing objects ignorable?
Model form: Are the link, distribution, covariance, time trend, and interaction structure defensible?
Positivity and overlap: Are all intended comparisons represented in relevant covariate regions?
Missingness and selection: Could inclusion depend on an unobserved outcome or model failure?
Multiplicity: How many outcomes, contrasts, thresholds, subgroups, and interim looks were considered?

An assumption cannot be proved by a non-significant diagnostic test. Combine plots, subject knowledge, acquisition logs, residual checks, negative controls, and sensitivity analyses.

4. Worked biological-imaging example

Suppose a QPI experiment compares a perturbation with control across four independent experiments, three wells per condition and hundreds of cells per well. The scientific outcome is a change in dry-mass accumulation. A defensible workflow keeps experiment and well identifiers, performs quality control without seeing condition labels where possible, and models or aggregates the hierarchy. Pooling every cell as independent would make uncertainty artificially small.

Start with a tidy table containing the fields experiment, well, field, cell, time, condition, the measured outcome, prespecified covariates, and QC flags. Plot raw replicate summaries and within-replicate contrasts. Apply Limits of detection and quantification only after its assumptions map onto this table.

5. Computation pattern

This compact example demonstrates reproducible simulation and interval estimation. Replace it with a model suited to this chapter; preserve the random seed, package versions, units, exclusions, and full model formula in the analysis record.

import numpy as np
from scipy import stats

rng = np.random.default_rng(42)
control = rng.normal(10.0, 1.1, 24)
treated = rng.normal(10.8, 1.2, 24)
result = stats.bootstrap((treated, control), lambda a, b: np.mean(a)-np.mean(b), paired=False, rng=rng)
print(result.confidence_interval)

For advanced models, use maintained implementations in statsmodels, PyMC, lifelines, scikit-learn, or a validated R package as appropriate. Confirm parameterisation because link functions, contrasts, variance estimators, and default reference levels differ across software.

6. Diagnostics and sensitivity analysis

Inspect raw data and fitted-versus-observed behaviour at every sampling level. Examine residual structure, influential groups, calibration, convergence, effective sample size, posterior predictive behaviour, or proportionality as appropriate. Refit plausible alternative specifications: robust variance, different correlation structures, transformed outcomes, alternative priors, exclusion rules, or matching tolerances.

A conclusion is more credible when its direction and scientifically relevant magnitude survive reasonable alternatives. If it does not, report model dependence rather than selecting the most favourable result.

7. Interpretation and reporting

Report the estimand, design, independent sample sizes, descriptive summaries, model formula, effect estimate, interval, diagnostics, sensitivity analyses, software versions, and limitations. Translate parameters back to scientific units. Avoid saying “no effect” solely because a threshold was not crossed, and avoid treating statistical significance as biological importance.

8. Common failure modes

Confusing technical observations with independent biological replication.
Choosing the model after inspecting which result is significant.
Ignoring interactions, nonlinearities, censoring, zero inflation, or temporal/spatial dependence.
Performing preprocessing or feature selection before cross-validation splits.
Reporting only a p-value, accuracy, or aggregate score without uncertainty and operating context.

9. Exercises

Draw the sampling hierarchy for the QPI example and identify the experimental unit.
Write the estimand in words and mathematical notation; then list assumptions needed to identify it.
Construct one diagnostic and one sensitivity analysis that could overturn the initial conclusion.

Answer outline

The treatment is assigned and independently repeated at experiment/well level, so cells are nested observations rather than independent treatment allocations. A suitable estimand could be the mean between-condition difference in replicate-level dry-mass accumulation over a prespecified interval. Diagnostics must address hierarchy and model form; sensitivity analysis should change a defensible modeling choice rather than search arbitrarily for significance.

10. References and next steps

Continue through the prerequisite links above, then use the related-note panel to compare neighboring methods before committing to an analysis.