When researchers in population and family health studies design a questionnaire to measure something like maternal stress, contraceptive knowledge, or family well-being, they face a deceptively simple question: is this tool actually measuring what we think it’s measuring? A scale could be neatly designed, consistently administered, and statistically reliable, yet still fail at its core job. This is where validity enters the picture. It is the quality that separates a meaningful measurement from a misleading one, and understanding it is essential for any student working with surveys, scales, or social indicators.
Table of Contents
What validity really means
Validity refers to the extent to which a research instrument actually measures what it claims to measure. In simple terms, it answers the question, “Are we measuring what we think we are measuring?” If a researcher builds a scale to assess women’s autonomy in household decisions but the items end up tapping into income levels instead, the scale lacks validity, no matter how carefully it is administered.
It is easy to confuse validity with reliability, but the two are quite distinct. Reliability is about consistency; a reliable instrument gives similar results when used repeatedly under similar conditions. Validity is about accuracy – whether those consistent results are actually capturing the intended concept. A faulty weighing scale that always shows a weight 10 kg higher than the true weight is reliable but not valid, as illustrated in the medical research literature. An instrument can therefore be reliable without being valid, but it cannot be truly valid without also being reliable.
In population and family health research, where we often study abstract phenomena like marital satisfaction, gender attitudes, or perceived health status, validity becomes even more critical. These concepts cannot be observed directly. Researchers must operationalise them through indicators, and the strength of those indicators determines whether the findings tell the truth about the population under study.
The main types of validity
Most methodology textbooks classify validity into three core types – content, criterion, and construct. Each addresses a different aspect of whether an instrument is truly fit for purpose.
Content validity
Content validity asks whether a measure covers the full range of the concept it is supposed to measure. It is about completeness and representativeness. If a researcher designs a questionnaire on adolescent reproductive health knowledge but only asks about menstruation, ignoring contraception, sexually transmitted infections, and consent, the tool will have poor content validity because it leaves out major dimensions of the construct.
Establishing content validity is largely a judgement-based process. Researchers consult existing literature, theoretical frameworks, and subject experts to ensure that every important dimension is captured by at least one item. Best-practice guidelines for scale development highlight the use of expert panels who rate each item on relevance, clarity, and adequacy, with revisions made until consensus is reached. This is particularly important for instruments measuring complex domains like family functioning, where missing even one dimension can distort the entire picture.
Criterion validity
Criterion validity examines how well a measurement correlates with another established measure or with a real-world outcome, ideally a gold standard. It is the most empirically grounded type of validity because it relies on comparison rather than expert judgement alone. Criterion validity is typically divided into two sub-types based on timing.
Predictive validity looks at how well an instrument forecasts a future outcome. For instance, a screening questionnaire designed to identify women at risk of postpartum depression has high predictive validity if women who score above the threshold actually go on to develop depressive symptoms in the months following childbirth. Methodology references describe this as a validation strategy where the criterion is measured after the test is administered.
Concurrent validity, in contrast, compares the new measure with an established criterion at the same point in time. If a short, community-friendly nutrition screening tool produces results similar to a detailed clinical dietary assessment carried out the same day, it has good concurrent validity. This is especially useful when researchers need a quicker, cheaper alternative to a lengthy gold-standard procedure for large field surveys.
Construct validity
Construct validity is the most conceptually demanding form of validity. It assesses whether an instrument truly captures the theoretical construct it is meant to represent. Because constructs like empowerment, social cohesion, or maternal stress cannot be observed directly, construct validity must be built up through multiple lines of evidence rather than a single test.
Two important sub-components support construct validity. Convergent validity shows that the new instrument correlates well with other measures that should theoretically be related to it; a new scale of women’s empowerment should correlate with established autonomy indices. Discriminant validity shows that it does not correlate with measures of unrelated concepts; an empowerment scale should not behave the same way as a measure of household income alone.
An example of how these types come together can be seen in the validation of the Indian version of the HLS-EU-Q16 health literacy questionnaire, where researchers tested item content with experts, examined construct validity through factor analysis, and assessed discriminant validity by comparing scores across education levels.
Approaches to establishing validity
Knowing the types of validity is only half the work. Researchers also need practical methods to actually demonstrate validity in their studies. Four widely used approaches are particularly relevant in population and family health research.
Logical validity
Logical validity, sometimes called face validity, is the most basic and intuitive approach. It involves a careful, reasoned examination of whether each item in the instrument appears to measure what it is intended to measure. The researcher uses theoretical understanding and common sense to check whether the questions logically fit the construct.
For example, a question like “How many meals do you skip in a typical week?” logically belongs in a survey on dietary habits, but a question about television viewing would feel out of place. Logical validity is subjective and considered a weaker form of evidence, but it is valuable in the early stages of instrument development for catching obvious mismatches before more rigorous testing begins.
Jury opinion
The jury opinion approach formalises expert judgement into a structured validation procedure. A panel of subject specialists – typically experienced researchers, clinicians, or field practitioners – independently reviews each item and rates it on relevance, clarity, and representativeness. Items that fail to receive sufficient consensus are revised or discarded.
A widely used refinement of this approach is the Delphi technique, in which experts rate items anonymously, receive summarised feedback, and rate them again over several rounds until ratings stabilise and consensus emerges. This method works well for sensitive or culturally complex topics in family health research, where individual biases can otherwise distort instrument design.
Known-group technique
The known-group technique is a clever empirical approach often used to support construct validity. It works by administering the instrument to two or more groups whose status on the construct is already well established, then checking whether the instrument can discriminate between them. As this method is commonly described, a valid measure should produce significantly different scores across groups known to differ on the trait being measured.
For instance, a new scale of parenting stress should produce higher average scores among parents of children with serious chronic illnesses than among parents of healthy children. If both groups score the same, something is wrong with the instrument. Likewise, a domestic violence screening tool should yield higher risk scores among women already known to be in abusive relationships compared with those in supportive ones.
Independent criteria
The independent-criteria approach is closely tied to criterion validity. Here, the researcher compares the new instrument against an external, independent, and well-accepted measure of the same construct. The independent criterion may be a gold-standard clinical assessment, an established scale, official records, or a respected biological indicator.
For example, the accuracy of a new self-reported household food security tool can be tested by comparing its results against detailed 24-hour dietary recall interviews conducted by trained nutritionists. A maternal depression screening questionnaire can be validated against a structured psychiatric interview. The closer the correlation between the new instrument and the independent criterion, the stronger the evidence for its validity.
Why validity matters in population and family health
Validity is not a technical decoration. In population and family health studies, it has real consequences for policies and lives. A poorly validated screening tool for malnutrition could miss children who need help, while a weak measure of women’s empowerment could lead programmes to claim success that does not exist in reality. Methodological reviews emphasise that validation is not a one-time task but an ongoing process, especially when tools are translated, adapted across cultures, or applied to new populations.
This becomes particularly important in diverse social contexts where family structures, gender norms, and health beliefs vary widely. A questionnaire about household decision-making that works well in urban nuclear families may completely misread joint families where extended kin play a major role. Cultural adaptation, repeated pilot testing, and rigorous re-validation are therefore essential whenever existing instruments are imported into new settings.
What do you think? If you were designing a new questionnaire to measure adolescent mental health in your community, which approach to validity would you prioritise first, and why? And how might the cultural context shape the way you decide whether your instrument is truly measuring what you think it measures?
References
- https://pmc.ncbi.nlm.nih.gov/articles/PMC10810057/
- https://www.frontiersin.org/journals/public-health/articles/10.3389/fpubh.2018.00149/full
- https://www.scribbr.com/methodology/types-of-validity/
- https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7827499/
- https://mmshub.cms.gov/measure-lifecycle/measure-testing/evaluation-criteria/scientific-acceptability/validity
- https://strokengine.ca/en/glossary/known-groups-method/
Leave a Reply