Type I and type II errors
- Concept
Type I error: seeing a difference that does not exist (false positive). Type II error: missing a difference that does exist (false negative).
What it is
Every hypothesis test ends in a decision that can go wrong in two ways. A type I error is rejecting H0 when it is true, with a maximum probability of α; a type II error is failing to reject H0 when it is false, with probability β. Cutting one without increasing the sample size usually increases the other, and the study’s power is 1 − β.
Worked example
- Data: an agency evaluates 100 new treatments; 20 truly work and 80 do not. Each trial uses α = 0.05 and 80% power.
- Of the 80 that do not work, 0.05 × 80 = 4 come out significant by chance: type I errors.
- The other 80 − 4 = 76 are correctly set aside (true negatives).
- Of the 20 that work, 0.80 × 20 = 16 come out significant (true positives).
- The other 20 − 16 = 4 miss significance: type II errors (β = 0.20).
- Of the 16 + 4 = 20 significant results, 4 are false: 20%.
- Reading: with α = 0.05, a share of positive findings can be false when few of the ideas tested are true.
The truth against the test’s decision
| Decision | H0 true (no effect) | H0 false (real effect) |
|---|---|---|
| Reject H0 | Type I error (α), false positive | Correct decision (power, 1 − β) |
| Do not reject H0 | Correct decision (1 − α) | Type II error (β), false negative |
Markers
- α
- β
- 1 − β
In practice
- A type I error can lead to adopting a useless treatment; a type II error can lead to abandoning a useful one.
- Small trials with a ‘negative’ result deserve caution: their β may be high.
Common pitfalls
- Confusing α with the p value: α is the threshold set beforehand; p is calculated from the data.
- Believing a negative study rules out an effect without checking its power and CI.
Test yourself
Think of the answer, then tap the question to see it.
What is a type I error?
Rejecting the null hypothesis when it is true: a false positive. Its maximum probability is α.
What is a type II error?
Failing to reject the null hypothesis when it is false: a false negative. Its probability is β.
If β = 0.10, what is the study’s power?
Power = 1 − β = 0.90, that is 90%.
What happens to β if α is lowered from 0.05 to 0.01 without changing the sample size?
β rises: the study loses power and is more likely to miss a real effect.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: What are statistical tests? (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012); Sample sizes required (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.