Validity and reliability
- Concept
Validity is measuring the true value; reliability is getting the same value when the measurement is repeated.
What it is
A measurement is valid (accurate) when, on average, it matches the true value, and reliable (precise, reproducible) when repeating it gives similar results. Lack of validity comes from systematic error, or bias, which shifts every measurement in the same direction. Lack of reliability comes from random error, which scatters them. A measurement can be reliable without being valid, but if it is very unreliable it cannot be valid for any single reading either.
Worked example
- Data: a water bath's reference value is 37.0 °C and it is measured 5 times with two thermometers.
- Thermometer A: 38.0, 38.1, 37.9, 38.0, 38.0. Sum 190.0, mean 38.0; range 0.2.
- Thermometer B: 36.4, 37.6, 37.0, 36.8, 37.2. Sum 185.0, mean 37.0; range 1.2.
- A has a systematic error of +1.0 °C but little scatter: reliable, not valid.
- B has no bias on average but a lot of scatter: valid on average, poorly reliable.
- Reading: A's bias is fixed by calibration; B improves only by averaging several readings or changing instruments.
The four cases of the target
| Case | Validity | Reliability | Main error |
|---|---|---|---|
| Darts grouped on the bullseye | High | High | None of note |
| Darts grouped away from the bullseye | Low | High | Systematic (bias) |
| Darts scattered around the bullseye | High on average | Low | Random |
| Darts scattered away from the bullseye | Low | Low | Both |
Markers
- validity
- reliability
- bias
- random error
In practice
- A poorly calibrated blood pressure monitor can give repeatable but consistently high readings: reliable, not valid.
- Agreement between observers (for example, kappa) measures the reliability of a clinical judgement, not its validity.
Common pitfalls
- Believing more measurements fix a bias.
- Confusing reliability with validity: two observers agreeing does not mean they are right.
- Using precision as a synonym for accuracy.
Test yourself
Think of the answer, then tap the question to see it.
A scale always reads 2 points too high, with nearly identical readings. Is it valid, reliable or both?
Reliable but not valid: it has systematic error with little scatter.
Which type of error is reduced by averaging many measurements?
Random error; systematic error stays in the average.
Does the kappa coefficient measure validity or reliability?
Reliability: agreement between observers beyond chance, not whether they match the truth.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: 2.1.1.3. Bias and Accuracy (NIST/SEMATECH e-Handbook of Statistical Methods, 2012); 2.1.1.4. Variability (NIST/SEMATECH e-Handbook of Statistical Methods, 2012); Interrater reliability: the kappa statistic (Biochemia Medica, 2012).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.