Understanding Types and Designs of Evaluation in Development Projects

Imagine a state government launches a massive campaign to reduce anaemia among adolescent girls. Crores are spent, posters go up, iron tablets are distributed. Two years later, someone asks: did it actually work? Without a proper evaluation framework, that question has no honest answer. Evaluation is what turns development spending into development evidence, and the type and design of evaluation chosen at the start decides whether the answer is trustworthy or just wishful thinking.

Why evaluation needs a type and a design 🔗

Many students confuse evaluation with monitoring, but the two are not the same. Monitoring is continuous tracking of inputs and activities, while evaluation is a periodic, deeper judgment of whether a programme is on track, working, or worth continuing. The UNICEF Evaluation Policy defines it as a systematic and objective assessment of an ongoing or completed project, looking at relevance, effectiveness, efficiency, impact and sustainability.

Two decisions sit at the heart of every evaluation plan. First, what type of evaluation is needed, which depends on the stage of the project. Second, what design will be used to compare what happened with what would have happened otherwise. Get either of these wrong, and the conclusions become shaky no matter how much data you collect.

Types of evaluation across the project life cycle 🔗

Development projects move through phases, and each phase asks a different question. The four most widely used types of evaluation map neatly onto these phases.

Baseline evaluation 🔗

A baseline evaluation is carried out before a programme begins. Its job is to capture the existing situation on the indicators the project hopes to change, so that later comparisons are possible. Without a baseline, any claim of improvement is essentially a guess.

For example, the National Family Health Survey functions as a recurring baseline for India’s health programmes, where each round provides reference values that the next round can be compared against. NFHS-1 in 1992-93 established the first nationally comparable data on fertility, contraception and child mortality, and every survey since has used the previous one as a benchmark.

Formative evaluation 🔗

Formative evaluation happens during the early or middle stages of implementation. It is diagnostic rather than judgmental. The aim is to spot problems early and feed the findings back into programme design so that course corrections can be made while there is still time. Formative evaluations typically assess feasibility, acceptability, reach and the quality of delivery rather than final outcomes.

A formative evaluation of a school deworming programme, for instance, might find that tablets are reaching schools but absent children are being missed, or that teachers are uncomfortable supervising distribution. These findings allow managers to redesign the delivery model before the full impact assessment is done.

Summative evaluation 🔗

Summative evaluation is conducted at the end of a programme cycle or at a major milestone. It asks the bottom-line question: did the programme achieve its intended outcomes? According to guidance from evaluation specialists, summative evaluations gather information from multiple sources to demonstrate overall effectiveness, and are usually the basis for reports to funders and decisions about scale-up.

If a maternal nutrition project set out to reduce low birth weight in a district, the summative evaluation would compare end-line data with the baseline to judge whether the target was met and at what cost. It is also the type of evaluation most often demanded by donors and ministries before they commit to a second phase.

Follow-up evaluation 🔗

Follow-up or ex-post evaluation takes place well after a project has ended, usually one to several years later. Its purpose is to check whether the gains achieved at the close of the project have been sustained, faded, or even reversed. Many interventions show strong results immediately after implementation but lose ground once external support is withdrawn, and only a follow-up can detect this.

A follow-up evaluation of a self-help group financial literacy programme, for example, might reveal that savings behaviour continued for two years but credit discipline eroded once handholding stopped. Such findings are critical for designing sustainability strategies in future projects.

Evaluation designs: how comparisons are built 🔗

Even with the right type of evaluation chosen, the conclusions depend on how the comparison is structured. Two broad design choices dominate development practice: pre-program versus post-program designs, and with-program versus without-program designs.

Pre-program versus post-program design 🔗

This is the most intuitive design. The same group is measured before the intervention and again after it. A pretest-posttest design tracks change within the participating group, which makes it relatively simple, affordable and easy to communicate.

The strengths are obvious. Programme managers get a clear picture of how participants moved on a particular indicator, and baseline data can also be used for planning. The limitations, however, are equally important. Without a comparison group, it is impossible to know whether the observed change was caused by the programme or by something else happening at the same time, such as a parallel government scheme, seasonal trends, or general economic improvement. As one methodological review notes, this design is best described as pre-experimental because confounding factors like maturation, history and natural growth cannot be ruled out.

This design is most appropriate when the intervention reaches the entire target population (so no comparison group is feasible), when budgets are tight, or when the goal is learning rather than rigorous attribution.

With-program versus without-program design 🔗

This design compares people or areas that received the intervention with similar people or areas that did not. The without-programme group is called a comparison or control group, and it serves as a stand-in for what would have happened to the intervention group in the absence of the project, often described as the counterfactual.

A study by the International Initiative for Impact Evaluation emphasises that establishing a credible counterfactual is the single most important step in claiming that a programme caused an observed change. Without it, the evaluator is comparing outcomes only to a hypothetical scenario.

When the comparison group is built through randomisation, the design becomes a randomised controlled trial, considered the gold standard for attribution. When randomisation is not feasible, evaluators use quasi-experimental approaches such as matched comparison groups, difference-in-differences, or propensity score matching. The strength of these designs is much stronger causal evidence. The trade-offs are cost, time, ethical concerns about denying services to a control group, and the technical expertise required.

Combining both: the pretest-posttest with comparison group 🔗

The most robust practical design used in population and family health studies combines the two. Both an intervention group and a comparison group are measured before and after the programme. Changes in the intervention group are then compared against changes in the comparison group, isolating the programme’s contribution from background trends. Methodological guides highlight that this design controls for most threats to internal validity, except attrition, and is widely used when full randomisation is not possible.

Choosing the right evaluation 🔗

There is no universally best evaluation type or design. The right choice depends on a careful reading of four things.

Project stage and purpose 🔗

If the project is still in design, a baseline is non-negotiable. If implementation is underway and managers want to fix problems, formative evaluation is the natural fit. If a funding cycle is closing, summative evaluation is expected. If the concern is whether benefits last, plan a follow-up.

The decision the evaluation must inform 🔗

A learning-oriented evaluation aimed at internal improvement can use lighter designs, including simple pre-post comparisons or qualitative methods. An accountability-oriented evaluation that will decide whether a scheme is scaled up nationally or shut down needs a stronger design with a credible comparison group, because the stakes of a wrong conclusion are much higher.

Resources and feasibility 🔗

Strong designs are expensive. A district-level randomised trial can cost crores and take three to five years. If the budget is modest, a well-executed pretest-posttest design with thoughtful qualitative triangulation is far more useful than a half-funded experimental design that collapses midway. The UNICEF evaluation framework explicitly recognises that the rigour of the design must be proportionate to the scale, complexity and decisions at stake.

Ethical and contextual constraints 🔗

Some interventions cannot ethically be withheld from a comparison group, for instance life-saving treatments or universal entitlements like immunisation. In such cases, evaluators must rely on phased rollouts, historical comparisons, or natural experiments. Cultural and political sensitivity also matters, especially when working on topics like adolescent reproductive health, gender-based violence or caste-based disparities, where data collection itself has to be designed with care.

Putting it together in practice 🔗

A well-evaluated project usually does not pick one type of evaluation; it uses several across the life cycle. A baseline establishes the starting point. Formative evaluation in the early phase keeps implementation honest. Summative evaluation at the end produces the headline findings for stakeholders. A follow-up a couple of years later checks for sustainability. Each layer adds a different kind of evidence, and together they tell the full story of a programme.

The choice of design then runs through all of these like a spine. If a comparison group is feasible, build it in from the baseline itself, because it is almost impossible to construct one credibly after the project has ended. If randomisation is not possible, plan for matched comparisons or phased rollouts early enough that they shape sampling and data collection. Documents like the OECD-DAC evaluation criteria on relevance, coherence, effectiveness, efficiency, impact and sustainability can then be used to interpret the evidence consistently across types and designs.

Good evaluation, in short, is not an afterthought tacked on at the end of a project. It is a research design decision made at the very beginning, revisited at each phase, and used to feed evidence back into the next round of programme planning.

What do you think? If you were designing the evaluation for a new adolescent anaemia programme in your district, which combination of evaluation types and designs would you prioritise, and what would you be willing to sacrifice if your budget was cut by half? How would you balance the demand for rigorous comparison groups against the ethical discomfort of withholding services from communities that need them?

References

1. https://www.unicef.org/evaluation/evaluation-policy
2. https://pmc.ncbi.nlm.nih.gov/articles/PMC10657051/
3. https://www.ncbi.nlm.nih.gov/books/NBK594901/
4. https://www.strategicpreventionsolutions.com/post/what-are-the-different-types-of-evaluation
5. https://cyfar.org/node/276
6. https://rforhr.com/pretestposttest.html
7. https://www.3ieimpact.org/sites/default/files/2019-01/wp10-impact-evaluation.pdf
8. https://sphweb.bumc.bu.edu/otlt/mph-modules/programevaluation/ProgramEvaluation7.html
9. https://www.unicef.org/evaluation/documents/unicef-evaluation-policy-2018
10. https://www.oecd.org/dac/evaluation/2754804.pdf