What is Evaluation? A Guide to Measuring Program Success

Every public health program – whether it’s a vaccination drive in a rural block of Bihar, a maternal nutrition scheme under POSHAN Abhiyaan, or a sanitation campaign in an urban slum – begins with good intentions and a budget. But how do we actually know if these programs worked? Did the money reach the right people? Did behaviour change? Did lives improve? This is where evaluation enters the picture. Far from being a bureaucratic formality, evaluation is the disciplined lens through which we measure whether a program delivered on its promises and what we should do differently next time.

Defining evaluation 🔗

At its core, evaluation is a systematic process of judging the merit, worth, or significance of a program. It is not a casual review or a year-end summary. It follows defined methods, uses credible evidence, and answers specific questions about outcomes and effectiveness.

The United States Agency for International Development (USAID), one of the most influential global development bodies, offers a widely accepted definition. According to its Evaluation Policy, evaluation is the systematic collection and analysis of information about the characteristics and outcomes of programs and projects as a basis for judgements, to improve effectiveness, and to inform decisions about current and future programming. USAID also clarifies that the purpose of evaluations is twofold – to ensure accountability to stakeholders and to learn in order to improve development outcomes.

This definition is important because it separates evaluation from two cousins that often get confused with it. Monitoring tracks ongoing activities and outputs in real time – for example, how many ORS packets were distributed last month. Audit checks financial and procedural compliance. Evaluation is broader: it asks whether the program achieved meaningful change, why it did or didn’t, and whether that change was worth the investment.

Why evaluation matters in population and family health 🔗

In population and family health work, programs often run for years and consume substantial public funds. Without evaluation, we would never know whether the National Family Health Survey (NFHS) cycles, family planning outreach, or adolescent health interventions are producing real demographic and health gains. Evaluation provides the evidence base that separates well-meaning activity from genuine impact, and it helps practitioners refine interventions to address persistent health inequities. As one review notes, evaluation contributes to our knowledge about the determinants of health issues as well as the most appropriate interventions to address them.

Types of evaluation 🔗

Before diving into the steps, it helps to know that evaluations are not all the same. Two broad categories shape how a study is designed.

Formative evaluation 🔗

This type takes place while a program is still being implemented. The purpose is to identify what is working, what isn’t, and to make mid-course corrections. Think of it as tasting the soup while it is still cooking. A formative evaluation of a new tuberculosis screening drive might reveal that ASHA workers are skipping certain hamlets, allowing the district to fix the gap before the campaign ends.

Summative evaluation 🔗

This is conducted after the program ends. It measures overall effects, impact, and whether stated objectives were met. Continuing the cooking analogy, it is the dinner guests giving their verdict after the meal. Summative evaluations often inform large policy decisions – whether to scale up, modify, or discontinue a program.

Within these categories, evaluators may also distinguish between process evaluation (was the program implemented as planned?), outcome evaluation (did short-term goals get met?), and impact evaluation (what long-term, attributable change occurred?). A rigorous impact evaluation often uses comparison groups or even randomised controlled trial designs to isolate cause and effect.

Steps in designing an evaluation 🔗

Designing a credible evaluation is a structured exercise. Whether you are a postgraduate student writing a dissertation or a district programme officer commissioning an external study, the workflow tends to follow the same logical sequence.

Step 1: Engage stakeholders and define the purpose 🔗

The first step is to clarify why the evaluation is being undertaken and who will use the findings. Funders, programme managers, frontline workers, and beneficiaries all have different interests. Engaging them early ensures the evaluation answers questions that actually matter. The National Institute of Justice notes that decisions about what data to collect flow directly from these stakeholder priorities.

Step 2: Write the evaluation proposal 🔗

A formal proposal is the blueprint of the entire study. A well-written evaluation proposal typically contains the following elements:

• Background and rationale: A description of the program, its goals, and why the evaluation is needed.
• Evaluation objectives and questions: Specific, answerable questions aligned with the programme’s objectives.
• Indicators: Measurable benchmarks that signal success or failure of each objective.
• Methodology: The design (formative, summative, mixed-methods), sampling strategy, and instruments such as questionnaires, interviews, or focus group discussions.
• Timeline: A realistic schedule of activities.
• Budget: Costs for fieldwork, personnel, analysis, and dissemination.
• Ethical safeguards: Plans for informed consent, confidentiality, and Institutional Ethics Committee clearance.

The Brown University Division of Research emphasises that a strong design first determines who will be studied and when, then selects the methodological approach and data collection instruments.

Step 3: Collect data 🔗

Once the proposal is approved, fieldwork begins. Data collection in population and family health studies usually draws on a mix of sources:

• Primary quantitative data through structured surveys, anthropometric measurements, or clinical records.
• Primary qualitative data through in-depth interviews, focus group discussions, and direct observation.
• Secondary data from sources like the Ministry of Health and Family Welfare, NFHS, the Health Management Information System (HMIS), and the Sample Registration System.

A national-level compendium on monitoring and evaluation in public health highlights mixed-methods approaches as particularly valuable because they triangulate evidence and produce more robust conclusions. Quality of data collection – training of enumerators, pretesting tools, supervision in the field – is what separates a publishable evaluation from a flawed one.

Step 4: Analyse the data 🔗

Quantitative data is processed using statistical software such as SPSS, Stata, or R. Common techniques include descriptive statistics, chi-square tests, regression models, and difference-in-differences analysis for impact evaluations. Qualitative data is coded thematically, often with tools like NVivo or Atlas.ti. The aim is to translate raw responses into evidence-based answers to the original evaluation questions.

Step 5: Write the evaluation report 🔗

The report is the most visible product of the entire exercise. A standard evaluation report includes:

• Executive summary with key findings and recommendations for busy decision-makers.
• Introduction covering the program background and evaluation purpose.
• Methodology describing design, sampling, instruments, and limitations.
• Findings organised around the evaluation questions, supported by tables, charts, and narrative.
• Discussion interpreting the findings against existing literature and context.
• Conclusions and recommendations that are specific, feasible, and prioritised.
• Annexures with tools, datasets, and ethics documentation.

A good report is written in clear language. If a frontline worker, a policymaker, and a researcher cannot all find what they need within it, the report has not done its job.

Step 6: Disseminate and use the findings 🔗

An evaluation that sits on a shelf is wasted effort. Findings must be shared through stakeholder workshops, policy briefs, journal articles, and presentations to ensure they actually influence the next round of decisions.

The role of evaluation in accountability 🔗

Accountability is arguably the strongest argument for investing in evaluation. Public health programs are funded by taxpayers or by international donors, and both groups have a legitimate right to know how their money is being used.

Accountability to funders and stakeholders 🔗

USAID’s own policy frames accountability as ensuring that funds are used efficiently, measuring effectiveness, disclosing findings, and using evaluation findings to inform budget decisions. The same logic applies to Indian programs financed by the central government, state governments, or bilateral donors. Without evaluation, agencies have no defensible answer when Parliament, the Comptroller and Auditor General, or the Supreme Court asks whether a flagship scheme delivered value for money.

Accountability to communities 🔗

Beyond financial reporting, evaluation also creates accountability to the people the program is meant to serve. A study published in the Indian Journal of Community Medicine argues that accountability is a core component of health-care reforms and an essential lever for universalising health services to the grassroots. When evaluation results are shared transparently, communities can hold service providers to higher standards and demand course corrections.

Guiding future decisions 🔗

The “learning” function of evaluation is what makes it forward-looking. Findings inform whether a pilot should be scaled, modified, or discontinued. They guide budget reallocations, training priorities, and even legislative changes. For example, evidence from successive NFHS rounds has directly shaped the design of Anaemia Mukt Bharat, Mission Indradhanush, and reforms in maternal health entitlements. As one protocol on development evaluations notes, evaluation contributes to cost-cutting and cost-effectiveness by clarifying what works and what does not – a critical input given the steady rise in public expenditure.

Building a culture of evidence 🔗

Finally, regular evaluation builds an organisational culture where decisions are grounded in evidence rather than instinct. When every program is expected to demonstrate results, designers plan more carefully, implementers monitor more rigorously, and policymakers become better consumers of data. Over time, this cultural shift is what transforms a health system from reactive to learning-oriented.

Common challenges in conducting evaluations 🔗

Despite its importance, evaluation in resource-constrained settings faces real obstacles. Budgets are often tight, baseline data may be missing, and political pressure can distort findings. Rural evaluations in particular can be hampered by geographical barriers, shortages of trained personnel, and difficulty in engaging stakeholders consistently. Strong ethical oversight, independent evaluators, and clear protocols help mitigate these risks. Pre-registering evaluation designs and publishing negative findings – not just success stories – also strengthens the integrity of the field.

What do you think? If you were asked to evaluate a maternal and child health program in your home district, which two outcomes would you choose to measure first, and why? And do you believe evaluation findings should be made public in full – even when they expose failures?

References

1. https://usaidlearninglab.org/evaluation/evaluation-toolkit/evaluation-policy-usaid
2. https://pmc.ncbi.nlm.nih.gov/articles/PMC4131424/
3. https://www.ferris.edu/administration/academicaffairs/vpoffice/Academic_Research/planning/prop_components/evaluation.htm
4. https://nij.ojp.gov/topics/articles/plan-program-evaluation-start
5. https://division-research.brown.edu/writing-evaluation-plan
6. https://main.mohfw.gov.in/
7. https://www.researchgate.net/publication/396162084_Monitoring_and_Evaluation_in_Public_Health_-_Concepts_Frameworks_Indicators_and_Sectoral_Applications
8. https://nap.nationalacademies.org/read/24617/chapter/8
9. https://pmc.ncbi.nlm.nih.gov/articles/PMC7467206/
10. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8356269/