Reliability vs Validity in Research: Differences, Types and Examples

Reliability is whether a measure gives consistent results each time you use it, and validity is whether it actually measures what it claims to measure. A bathroom scale that always reads two kilograms heavy is reliable, because it gives the same wrong number every morning, but it is not valid.

You need both for results anyone can trust. This page sets the two side by side, walks through the main types of each, shows how to report them in a methods section, and ends with how qualitative researchers handle the same question.

TL;DR

• Reliability is about consistency: the same result across time, raters, items or test versions.

• Validity is about accuracy of meaning: whether the scores reflect the concept you care about.

• A measure can be reliable without being valid, but an unreliable measure cannot be valid.

• Qualitative studies use different criteria, such as credibility and dependability, rather than borrowing these terms.

Reliability vs validity: key differences

The APA Dictionary of Psychology defines reliability as the degree to which an instrument is free of random error, so it yields the same results when applied again to the same sample. It describes validity as how well evidence and theory support the conclusions you draw from an assessment.

What are the types of reliability?

Each type checks consistency from a different angle, and you only need the ones that fit how your measure is used.

• Test-retest reliability: give the same measure to the same people twice and correlate the scores. The APA Dictionary calls retest reliability an estimate of how stable the construct is over time, so choose a gap short enough that the trait itself should not have changed.

• Inter-rater reliability: have two or more trained people score the same thing independently, then check how closely they agree. It matters whenever a judgment is involved, such as coding interview transcripts or grading essays. Agreement is usually reported with a statistic such as Cohen’s kappa or an intraclass correlation.

• Internal consistency: check whether the items on one scale hang together, as they should if they all tap the same underlying thing. Cronbach’s alpha is the usual statistic.

• Parallel forms reliability: build two versions of a test from the same blueprint, give both to one group and correlate the scores. The dictionary files this under alternate-forms reliability. It is useful when people must sit a test more than once, as with a make-up exam.

A word on alpha. Many authors treat 0.70 as a minimum, but that is a convention rather than a rule. Tavakol and Dennick (2011) note that reported acceptable values range from 0.70 to 0.95, and that a very high alpha can mean some items simply repeat each other. Report the value you found and say what it means for your purpose.

What are the types of validity?

The first four types below are about a measure. The last two are about a whole study.

• Construct validity: whether the instrument captures the concept it is meant to. In the APA Dictionary’s example, a new aggression questionnaire should assess aggression, not assertiveness or social dominance.

• Content validity: whether the items cover the full range of what you are measuring. A Grade 9 algebra test that only asks about linear equations would fall short.

• Criterion validity: whether scores line up with an established standard, either at the same time (concurrent) or later (predictive), as the dictionary’s entry describes.

• Face validity: whether the items look appropriate to the people taking them. It helps participants take a measure seriously, but the dictionary warns that a test can have face validity yet lack empirical validity.

• Internal validity: whether the study’s design lets you draw sound conclusions within it, especially about cause and effect.

• External validity: whether the findings generalize beyond your sample to other people, places or times.

Internal validity is where bias in a study’s design shows up, through problems such as uneven groups, missing outcome data or unblinded assessors. When you appraise someone else’s study for a review, our risk of bias assessment tool drafts a risk-of-bias assessment of one study using RoB 2, ROBINS-I, QUADAS-2, the Newcastle-Ottawa Scale or a JBI checklist. Each result comes as a draft for you to review and confirm.

Can a measure be reliable but not valid?

Yes, and it happens more often than you might think. Consistency only tells you the measure is stable, not that it points at the right target.

• A miscalibrated scale gives the same weight every time, but every reading is off by the same amount.

• A “critical thinking” test made of long, wordy questions may produce very stable scores that mostly reflect reading speed.

• An interview rubric that two raters apply identically can still miss the skill it was written to judge.

The reverse does not hold. If scores jump around for reasons unrelated to the concept, they cannot track that concept well, which is why reliability is usually treated as a precondition for validity.

How do you show reliability and validity in a methods section?

1. Prefer an established instrument. Name it, cite the paper that developed or validated it, and say whether that work used a population like yours.

2. Check reliability in your own data. Reliability belongs to scores from a particular sample, so report your own alpha, retest correlation or rater agreement alongside the published figure.

3. Pilot anything new or adapted. Try the instrument with a small group like your participants, revise unclear items, and describe what you changed. If you translated it, say how.

4. State the evidence for validity. Mention expert review of the items, comparison with an established measure, or the patterns you expected with related and unrelated variables.

5. Name the threats you could not remove. Saying so is stronger than ignoring them.

If you are still drafting the plan itself, the AnswerThis methodology generator organizes the plan you supply into five sections, design, participants, procedure, measures and analysis, with placeholders for details you have not decided. Then add the reliability and validity evidence for each measure from your sources. You get three free runs without signing up.

What do reliability and validity look like in different fields?

Psychology: a new questionnaire

A team builds a 12-item scale on study burnout. They check internal consistency with alpha, give it again two weeks later for test-retest reliability, and show construct validity by correlating it with an established burnout measure and finding a weak link with an unrelated trait.

Nursing: a risk assessment scale

Two nurses score the same patients on a pressure injury risk scale without seeing each other’s ratings, which tests inter-rater reliability. Predictive validity asks whether patients rated high risk actually develop more injuries over the following weeks.

Education: an end-of-term exam

A chemistry department maps every question to the syllabus to support content validity. For students who miss the exam, it writes a second paper from the same blueprint, and a trial run with one class checks parallel forms reliability.

Laboratory science: a measuring instrument

In the lab, the same ideas show up as precision and accuracy. Repeated readings of one buffer solution on a pH meter show how consistent it is, while checking it against a certified reference solution shows whether those readings are correct. Regular calibration protects both.

For fuller study designs built around measures like these, see our examples of research methodologies.

How is quality judged in qualitative research?

Interview and observation studies do not produce scores, so “reliability” and “validity” fit them awkwardly. Many qualitative researchers use the trustworthiness criteria from Lincoln and Guba’s Naturalistic Inquiry (SAGE, 1985). Korstjens and Moser (2018) summarize them and the strategies that support each:

• Credibility: confidence that the findings truly reflect participants’ data, supported by prolonged engagement, persistent observation, triangulation and member checks.

• Transferability: how far findings might apply in other settings, which you help readers judge through thick description of your context.

• Dependability: the stability of findings, shown through an audit trail of the decisions you made.

• Confirmability: whether other researchers could confirm that the interpretations come from the data rather than your own assumptions, also supported by the audit trail and reflexive notes.

These are a different standard, not a weaker one. Judge a qualitative study by them, not by sample size or statistics.

What are the most common mistakes?

• Reporting only the published reliability of a questionnaire and never checking it in your own sample.

• Treating a high alpha as proof of validity. Alpha says the items agree with each other, not that they measure the right concept.

• Relying on face validity because the items “look right.”

• Adapting or translating an instrument and still citing the original validation as if nothing changed.

• Mixing up the validity of a measure with the internal or external validity of the study.

FAQs

Is reliability or validity more important?

Validity is the goal, since a measure is only useful if it captures the right concept. Reliability comes first in practice, because inconsistent scores cannot be valid.

What is a good Cronbach’s alpha?

There is no single cut-off. Values of 0.70 and above are a common convention, very high values can signal redundant items, and what counts as good depends on how the scores will be used.

Is face validity enough?

No. It shows that items seem relevant, but a measure can look right and still fail to capture the concept. Back it with expert review or comparison with other measures.

Do I need to check reliability if I use a published questionnaire?

Yes. Cite the published evidence and also report reliability in your own sample, since a scale that worked well in one population may not work as well in another.

Can qualitative research be reliable and valid?

Qualitative researchers usually speak of trustworthiness instead, judged by credibility, transferability, dependability and confirmability. Each has its own strategies, such as member checks and an audit trail.

Once your measures and their evidence are settled, use the methodology generator to organize the rest of your plan into a draft you can revise.