Questionnaire validation
- Concept
Checking that a questionnaire measures consistently (reliability) what it claims to measure (validity) and picks up change.
What it is
Validating a questionnaire means gathering evidence that its scores are reliable, valid and responsive in a given population. Reliability covers internal consistency (Cronbach's alpha), stability over time (test-retest) and agreement between observers. Validity covers content, construct and criterion validity. An instrument validated in another language or country is not simply translated: it needs cross-cultural adaptation and a new check.
Key facts
- Cronbach's alpha = [k / (k − 1)] × (1 − sum of item variances / variance of the total score), with k items.
- Reliable is not the same as valid: a questionnaire can always give the same result and still measure something else.
Worked example
- Data: a 4-item scale given to 120 people. Item variances: 1.0, 1.2, 0.8 and 1.0. Variance of the total score: 10.0.
- Sum of the item variances: 1.0 + 1.2 + 0.8 + 1.0 = 4.0.
- Alpha = [4 / (4 − 1)] × (1 − 4.0 / 10.0) = 1.333 × 0.60 = 0.80.
- Reading the alpha: 0.80 is acceptable internal consistency (≥ 0.70) with no sign of redundancy (> 0.95).
- Test-retest: 40 people answer again after 2 weeks; ICC = 0.85, good stability.
- Criterion: against pill counts, the score correlates at 0.62, a moderate correlation.
- Reading: the scale is reliable and shows criterion validity in this sample; to use it in another country it would need cross-cultural adaptation.
Measurement properties of a questionnaire
| Property | What it shows | How it is assessed |
|---|---|---|
| Internal consistency | The items measure the same thing | Cronbach's alpha |
| Test-retest | Stability over time | ICC; kappa for categorical answers |
| Inter-rater | Agreement between interviewers | ICC or kappa |
| Content validity | Covers every aspect of the concept | Judgement of experts and patients |
| Construct validity | Behaves as theory predicts | Expected correlations, factor analysis |
| Criterion validity | Agrees with a reference standard | Correlation, sensitivity and specificity |
| Responsiveness | Detects improvement or worsening | Change in score before and after |
Markers
- Cronbach's α
- ICC
- Kappa
In practice
- Many quality-of-life, depression and adherence scales have versions validated in Colombia: use the validated version, not a home-made translation.
- A score with no proven validity in the study population weakens the whole result.
Common pitfalls
- Translating a questionnaire and using it without adaptation or a new validation.
- Presenting a high alpha as proof of validity.
- Running test-retest over an interval so long that the condition has already changed.
Test yourself
Think of the answer, then tap the question to see it.
A 5-item scale has a sum of item variances of 5 and a total variance of 15. What is alpha?
(5/4) × (1 − 5/15) = 1.25 × 0.667 ≈ 0.83.
What does a Cronbach's alpha of 0.97 suggest?
That some items are redundant: they ask almost the same thing.
What does test-retest assess?
Stability: whether the questionnaire gives the same result when the condition has not changed.
What are the steps of a cross-cultural adaptation?
Independent translation, synthesis, back-translation, expert committee review and cognitive testing with people from the population.
Connections
Educational content on epidemiological and statistical methods; the examples use made-up numbers to teach the calculation. Not a clinical or legal guide: current Colombian rules are in the official sources.
Sources: COSMIN Taxonomy of Measurement Properties (COSMIN, 2010); Collecting Data (Centers for Disease Control and Prevention (The CDC Field Epidemiology Manual), 2024).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.