Statistical vs clinical significance
- Concept
A statistically significant result does not mean it matters to the patient. The size of the effect has to be checked.
What it is
Statistical significance asks whether a difference is hard to explain by chance; clinical significance asks whether it is large enough to matter. With huge samples tiny effects reach very small p values, and with small samples important effects may miss significance. The minimal important difference (MID), the smallest change patients perceive as beneficial, is the benchmark for judging the confidence interval.
Worked example
- Data: on a 0 to 100 point pain scale, the MID is 10 points. A very large trial enrols 10 000 people per group.
- Observed difference = 2 points, with SD = 20 in each group.
- SE = 20 × √(2 / 10 000) = 20 × 0.0141 = 0.28 points.
- z = 2 / 0.28 = 7.1, so p < 0.001: highly significant.
- 95% CI = 2 ± 1.96 × 0.28 = 1.4 to 2.6 points (rounded).
- The whole CI lies far below the MID of 10 points.
- Reading: the effect is almost surely real, but too small for patients to notice.
Four possible combinations
| Result | Statistically significant? | Clinically important? | Example CI (MID = 10) |
|---|---|---|---|
| Small, precise effect | Yes | No | 1.4 to 2.6 |
| Large, precise effect | Yes | Yes | 12 to 18 |
| Uncertain effect | No | Unknown | −3 to 20 |
| No important effect | No | No | −2 to 3 |
Markers
- MID
- 95% CI
- p
Common pitfalls
- Announcing a treatment as effective only because p < 0.05.
- Dismissing a potentially important effect because a small study gave p > 0.05.
- Presenting only the relative risk reduction, which makes a small absolute benefit look large.
Test yourself
Think of the answer, then tap the question to see it.
Why can a very small effect be statistically significant?
Because with very large samples the standard error is tiny, so any non-zero difference gives a small p value.
What is the minimal important difference?
The smallest change in an outcome measure that patients perceive as beneficial and that would justify a change in management.
A 95% CI for the difference runs from 1 to 3 points and the MID is 8. How is it read?
The effect is statistically significant (it excludes 0) but of little clinical importance: the whole interval lies below the MID.
Which measures help judge the clinical importance of a trial?
The effect size with its CI, the absolute risk reduction and the number needed to treat, set against the MID.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: Chapter 15: Interpreting results and drawing conclusions (Cochrane, 2024); What is the relationship between a test and a confidence interval? (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.