Psychological testing: reliability, validity and norms
- Concept
A good test gives the same result when nothing has changed (reliable) and truly measures what it claims to measure (valid).
What it is
Psychological tests turn behavior, abilities or symptoms into scores. To be useful, a test must be standardized (given and scored the same way every time), reliable (consistent) and valid (accurate for its purpose), and its scores must be compared with norms from a representative reference group. Reliability is necessary but not enough: a test can be consistently wrong. These same ideas apply to rating scales, screening questionnaires and clinical examinations.
Key facts
- Test-retest reliability is best reported as an intraclass correlation coefficient; inter-rater agreement often as kappa; internal consistency as Cronbach's alpha.
- Criterion validity is concurrent when compared with a reference standard at the same time, and predictive when compared with a later outcome.
- Face validity is whether a test looks, to the person taking it, as if it measures what it claims; it is the weakest form.
- Norms must match the person tested: a test normed in another country, language or generation can mislead.
- Scores drift upward over generations (the Flynn effect), so old norms overestimate performance.
- For screening tools, sensitivity and specificity describe validity against a diagnostic reference.
Reliability and validity, with an example
| Property | Type | Question it answers | Example |
|---|---|---|---|
| Reliability | Test-retest | Same result when repeated? | A memory test gives similar scores two weeks apart in a stable patient |
| Reliability | Inter-rater | Do two examiners agree? | Two nurses score the same patient's agitation scale almost identically |
| Reliability | Internal consistency | Do the items measure the same thing? | All items of a sadness scale rise and fall together |
| Validity | Content | Does it cover the whole domain? | A depression scale asks about sleep, appetite and energy, not only mood |
| Validity | Construct | Does it measure the concept in theory? | An anxiety scale correlates with other anxiety measures, less with unrelated traits |
| Validity | Criterion (concurrent) | Does it agree with a reference now? | A screening questionnaire agrees with a structured diagnostic interview |
| Validity | Criterion (predictive) | Does it predict a later outcome? | An entrance exam score predicts first-year grades |
Clinical notes
- A clock that always runs ten minutes fast is perfectly reliable and never valid: the same logic applies to a badly calibrated questionnaire.
- Test results are interpreted with education, language, culture, sensory limits and mood of the person in mind.
Test yourself
Think of the answer, then tap the question to see it.
What is the difference between reliability and validity?
Reliability is consistency of results; validity is whether the test measures what it is meant to measure.
Two psychologists score the same interview and agree closely. Which property is shown?
Inter-rater reliability.
Can a test be valid without being reliable?
No. Reliability is necessary for validity, but a reliable test can still be invalid.
A test score predicts later job performance. Which type of validity is this?
Predictive criterion validity.
What are norms in psychological testing?
Reference scores from a large, representative group, used to judge how one person's score compares with others.
Connections
Educational content on behavior and the clinical relationship; not a diagnostic or treatment guide and no substitute for assessment by a health professional. If you or someone near you is at risk, seek help now from an emergency service or a mental-health helpline.
Sources: A primer on reliability testing of a rating scale (Indian Journal of Psychiatry (PubMed Central), 2025); Psychometrics: Trust, but Verify (Anesthesia & Analgesia (PubMed Central), 2019); Assessment of Specific Learning Disabilities and Intellectual Disabilities (Assessment (PubMed Central), 2023).
Spotted a mistake?We are always improving for the community. Tell us what is wrong and our team will review it.Report a mistake
Representative model, for educational purposes only.