Monitoring and Evaluation

The Importance of Publishing Evaluations

Evaluations are done much more frequently these days than a few years ago. Donors and management both appreciate the reassurance and lessons-learned that evaluations can provide.

But the quality of evaluations has not markedly improved. Some are excellent, but almost anyone with field experience can become an evaluator. Nobody verifies the evaluator’s competence: it’s based on reputation. And when an aid agency is desperate to find someone to do an evaluation at very short notice, they’ll hire almost anyone.

Evaluations need to be planned, and are best done both during and after an aid programme. But planning ahead for evaluations is not the greatest strength of aid agencies. They often only seem to think of them once the programme is over, when many staff have departed, and when memories have faded.

ALNAP (see below) has taken the lead in trying to learn systematically from evaluations. It is also trying to help improve their quality.

Pleasingly, there is an increasing tendency to publish evaluations. UNHCR took the brave step of publishing the evaluation on its Kosovo emergency operation, even though it contained some serious criticisms. Type “Kosovo Evaluation” into the search box on www.unhcr.org and you’ll see it there: “The Kosovo refugee crisis: an independent evaluation of UNHCR’s emergency preparedness and response.”

The result was a significant boost to UNHCR's reputation, especially since UN agencies are not often keen to hear criticism. It was seen as a turning-point.

The way forward is surely for all aid agencies to follow UNHCR’s example and publish all their evaluations.

ALNAP

The Active Learning Network on Accountability and Performance in Humanitarian Action (ALNAP) is a leading network which gathers, collates, comments on and disseminates evaluations of humanitarian relief programmes. Its website is at www.alnap.org. The lessons from past evaluations are an interesting read.

It defines its role more broadly than evaluation: improving learning, accountability and performance across the humanitarian sector. One of the interesting and useful things that it does is to synthesise the findings from a large number of evaluations each year, and report them in its annual report.

Evaluating Humanitarian Action using the OECD-DAC Criteria, by Tony Beck, was recently (2006) published by ALNAP and is downloadable from their website. From the home page click on Evaluation. See below for more on OECD-DAC.

ODI Good Practice guide on evaluation

"Evaluating Humanitarian Assistance Programmes in Complex Emergencies" is the title of the Overseas Development Institute Good Practice Review, No 7. It is available at www.odihpn.org (click on publications, then select good practice reviews from the drop-down menu).

OECD DAC Guidance on Evaluation

The OECD (Organisation for Economic Co-Operation and Development) Development Assistance Committee (DAC) has published Guidance for Evaluating Humanitarian Assistance in Complex Emergencies. This document is widely used. It is at www.the-ecentre.net/resources/e_library/doc/OECD.pdf or obtainable through a search on Google if its web address changes in future.