Agreement for continuous measures: ICC and Bland-Altman
- Measure
Two devices can correlate almost perfectly and still not give the same values. That is what the ICC and Bland-Altman are for.
What it is
To know whether two methods or observers give the same numbers, correlation is not enough: it only shows whether they rise and fall together. The intraclass correlation coefficient (ICC) estimates what share of the total variation comes from real differences between subjects rather than measurement error. The Bland-Altman method analyses the differences between the two methods: their mean (the bias) and the 95% limits of agreement, which are judged against a clinical criterion set in advance.
Formula and reading
- Difference d = method B − method A for each subject; bias = mean of d.
- 95% limits of agreement = bias ± 1.96 × SD(d).
- ICC = σ² subjects / (σ² subjects + σ² error); it depends on the between-subject variation of the sample.
Worked example
- 6 patients, new blood pressure monitor against the reference (mmHg). Differences (new − reference): +4, −2, +6, +2, 0, +8.
- Bias = (4 − 2 + 6 + 2 + 0 + 8) / 6 = 18/6 = +3 mmHg.
- Deviations from the mean: 1, −5, 3, −1, −3, 5; sum of squares 70; SD = √(70/5) = √14 ≈ 3.74.
- Limits: 3 ± 1.96 × 3.74 = 3 ± 7.3, that is, from −4.3 to +10.3 mmHg.
- Criterion set in advance: differences of ± 5 mmHg are acceptable. The upper limit exceeds it.
- Reading: the device reads 3 mmHg high on average and may be off by up to 10; it cannot replace the reference (6 patients are few).
What each tool measures
| Tool | Question it answers | Limitation |
|---|---|---|
| Correlation (r) | Do they rise and fall together? | Ignores systematic bias; grows with the range of values |
| ICC | What share of the variation is real and not error? | Depends on sample heterogeneity; there are several formulas |
| Bland-Altman | How much do they differ, in the units of measurement? | Needs an external clinical criterion to decide |
| Kappa | The same idea for categorical variables | Not for continuous variables |
Markers
- ICC
- Bland-Altman
- Bias
- Limits of agreement
How to read it
- When validating a new device (blood pressure monitor, oximeter, thermometer), report bias and limits, not only r.
- Check whether the differences grow with the level measured: if so, analyse relative or log differences.
- State which form of ICC was used (absolute agreement or consistency; single rater or average).
Common pitfalls
- Concluding agreement because r = 0.95 or because the correlation has p < 0.05.
- Using a non-significant paired t test as proof of agreement: with few subjects it is rarely significant.
- Comparing ICCs from samples with very different ranges: a heterogeneous sample gives a higher ICC.
Test yourself
Think of the answer, then tap the question to see it.
Differences with mean +3 and SD 3.74. What are the 95% limits of agreement?
3 ± 1.96 × 3.74 = from −4.3 to +10.3 in the same units.
Why does a high correlation not prove agreement?
Because it measures a linear relationship, not equality: one method can always read higher and still correlate almost perfectly.
Between-subject variance 90 and error variance 10. ICC?
90 / (90 + 10) = 0.90.
Who decides whether Bland-Altman limits are acceptable?
A clinical or analytical criterion set before the study, not the statistics.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: Understanding Bland Altman analysis (Biochemia Medica, 2015); 2.1.1.4. Variability (NIST/SEMATECH e-Handbook of Statistical Methods, 2012); One-way ANOVA overview (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.