Critical appraisal of a paper
- Concept
Appraising a study means asking three things: is it valid, what did it find and how precisely, and does it apply to my patient?
What it is
Critical appraisal judges a paper in three steps. Internal validity: whether design and conduct avoid bias (randomisation, allocation concealment, blinding, losses, intention-to-treat analysis). Results: the size of the effect, its precision (CI) and its translation into absolute numbers. Applicability (external validity): whether the patients, intervention and outcomes resemble one's own setting and whether benefit outweighs harms and costs.
Worked example
- Data (made up): a trial assigns 1 200 adults aged 45 to 60 to a rehabilitation programme or usual care; outcome, readmission at 1 year.
- Validity: randomisation and concealed allocation; losses of 18 % in one arm and 6 % in the other: a warning, intention to treat is a must.
- Results: readmission 9 % with the programme and 12 % with usual care.
- RR = 9 / 12 = 0.75; relative reduction 25 %; ARR = 12 % − 9 % = 3 %.
- NNT = 1 / 0.03 = 33.3; rounded up to 34 patients for 1 year.
- Reported 95 % CI for the RR: 0.60 to 0.94; it excludes 1, a benefit with acceptable precision.
- Applicability: your patient is 82 and lives alone, outside the studied range: the effect may differ.
- Reading: a probably valid study with a warning about losses, a moderate benefit (NNT 34) and doubtful applicability to this patient.
Three questions for appraising a trial
| Question | What to check | Warning sign |
|---|---|---|
| Is it valid? | Randomisation, concealment, blinding, losses, intention to treat | Groups differ at baseline, unequal losses |
| What did it find? | RR, ARR, NNT, 95 % CI | Very wide CI, relative effects only |
| Does it apply here? | Population, intervention, outcome, benefit-harm balance | Very different patients, surrogate outcomes |
Markers
- Validity
- Results
- Applicability
- ITT
In practice
- With practice, appraising a trial takes a few minutes if you go straight to methods, table 1, the flow diagram and the absolute results.
- An abstract usually highlights the relative effect: computing the ARR and NNT changes the impression of benefit.
Common pitfalls
- Reading only the abstract and the authors' conclusions.
- Confusing statistical significance with clinical importance.
- Applying the result to patients outside the inclusion criteria without thinking about it.
Test yourself
Think of the answer, then tap the question to see it.
What are the three basic questions of critical appraisal?
Are the results valid? What are they, and how precise? Can they be applied to my patient?
A trial loses 18 % of one arm and 6 % of the other. Why does this matter?
Because unequal losses can break the comparability created by randomisation and bias the result.
A treatment lowers an event from 12 % to 9 %. What is the NNT?
ARR = 3 %; NNT = 1 / 0.03 = 33.3, rounded up to 34.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: Chapter 15: Interpreting results and drawing conclusions (Cochrane, 2026); CONSORT 2025 Statement: updated guideline for reporting randomised trials (EQUATOR Network, 2025).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.