P value
- Concept
The p value says how surprising the data would be if there really were no difference. It is not the probability that the effect is real.
What it is
The p value is the probability of a result as extreme as, or more extreme than, the one observed if the null hypothesis were true. A small p value means the data fit poorly with H0. It does not measure the size of the effect, its clinical importance or the probability that H0 is true, which is why it is always read together with the confidence interval.
Worked example
- Data: in a trial, systolic pressure falls 4 mmHg more with treatment A than with B; the SE of the difference is 2 mmHg.
- z = 4 / 2 = 2.0. The two-sided p value for z = 2.0 is about 0.046.
- 95% CI = 4 ± 1.96 × 2 = 0.08 to 7.92 mmHg: it excludes 0, just as p < 0.05.
- If the study were smaller and the SE were 2.5 mmHg: z = 4 / 2.5 = 1.6 and p ≈ 0.11.
- Its 95% CI = 4 ± 4.9 = −0.9 to 8.9 mmHg: it includes 0, just as p > 0.05.
- The observed difference (4 mmHg) is the same in both scenarios; only the precision changes.
- Reading: p ≈ 0.11 does not show that A and B are equal; the CI shows the difference could be zero or nearly 9 mmHg.
What the p value is and is not
| Statement | Correct? | Why |
|---|---|---|
| p is the probability of data this extreme or more if H0 were true | Yes | That is the definition |
| p is the probability that H0 is true | No | p is calculated assuming H0 is true |
| p = 0.001 means a bigger effect than p = 0.04 | No | p also depends on sample size |
| p > 0.05 proves there is no effect | No | Power may be lacking; look at the CI |
| A two-sided p < 0.05 matches a 95% CI that excludes the null | Yes | They are two sides of the same calculation |
Markers
- p
- α
- NS
Common pitfalls
- Treating 0.05 as a magic line: p = 0.049 and p = 0.051 carry almost the same evidence.
- Running many analyses and reporting the one with p < 0.05 (multiple comparisons).
- Reporting only the p value without the effect size and its CI.
Test yourself
Think of the answer, then tap the question to see it.
What exactly does a p value of 0.03 mean?
If H0 were true, there would be a 3% chance of observing a result as extreme as, or more extreme than, the one obtained.
Does a p value of 0.001 mean the effect is clinically important?
No. It only shows the data fit poorly with H0; clinical importance is judged from the effect size and its CI.
A study finds p = 0.30. Has it shown that the treatment does not work?
No. H0 is not rejected, but the study may have had little power; the CI shows which effects remain compatible with the data.
If the 95% CI of an RR runs from 1.1 to 1.8, what two-sided p value is expected?
Below 0.05, because the interval excludes 1, the null value for a ratio.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: Critical values and p values (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012); What is the relationship between a test and a confidence interval? (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012); Chapter 15: Interpreting results and drawing conclusions (Cochrane, 2024).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.