Choosing a statistical test
- Concept
The test is chosen with four questions: what type of outcome, how many groups, independent or paired, and whether the data are normal.
What it is
Choosing the right statistical test is one of the most common exam questions and one of the most common errors in papers. The choice depends on the type of outcome variable (categorical, numerical or time to an event), the number of groups, whether the measurements are independent or paired, and whether numerical data are roughly normally distributed. With those four answers there is nearly always one sensible option.
Function
- 1. Is the outcome categorical, numerical or a time to an event?
- 2. How many groups are compared: two, or three or more?
- 3. Are the measurements independent or paired (same people, matched pairs)?
- 4. If numerical: is the distribution roughly normal, or is it ordinal or skewed?
Worked example
- Data: a study of 120 patients in 3 groups of 40 measures systolic pressure, a pain scale, cure yes/no and time to relapse.
- Systolic pressure (numerical, normal) in 3 independent groups: one-way ANOVA.
- Pain (ordinal) in 3 groups: Kruskal-Wallis.
- Cure yes/no in 3 groups: chi-square on a 3 × 2 table, with (3 − 1) × (2 − 1) = 2 degrees of freedom.
- Pressure in the same 40 patients of group A before and after: paired t test.
- Cure yes/no in the same patients under two successive regimens: McNemar.
- Time to relapse with losses to follow-up: Kaplan-Meier curves and the log-rank test.
- Reading: one study uses six different tests because each outcome and each design asks a different question.
Which test to use by outcome and design
| Outcome | 2 independent groups | 2 paired measurements | 3 or more groups |
|---|---|---|---|
| Numerical, normal | Independent t test | Paired t test | ANOVA |
| Ordinal or skewed numerical | Mann-Whitney | Wilcoxon signed-rank | Kruskal-Wallis |
| Categorical (proportions) | Chi-square or Fisher | McNemar | Chi-square |
| Time to an event | Log-rank (Kaplan-Meier) | Methods for paired data | Log-rank or Cox model |
Common pitfalls
- Using a test for independent groups on paired data (power is lost and the result is wrong).
- Running several t tests between pairs of groups instead of one ANOVA (type I error inflates).
- Using chi-square with very small expected counts instead of the Fisher exact test.
Test yourself
Think of the answer, then tap the question to see it.
Which test compares mean cholesterol between two independent groups with normal data?
Student’s t test for independent samples.
Which test compares an ordinal scale across three groups?
Kruskal-Wallis, the non-parametric alternative to ANOVA.
Which test compares proportions measured twice in the same people?
The McNemar test, which uses only the discordant pairs.
A 2 × 2 table has an expected count of 3. Which test is suitable?
The Fisher exact test, because chi-square is unreliable with expected counts below 5.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: What are statistical tests? (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012); Principles of Epidemiology | Lesson 2 - Section 8 (Centers for Disease Control and Prevention, 2012); How can we compare the results of classifying according to several categories? (NIST/SEMATECH e-Handbook of Statistical Methods, National Institute of Standards and Technology, 2012).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.