Book Review: Impact Evaluation in Practice

This is an incredibly earth-bound book about (results-based) impact evaluation. Titled Impact Evaluation in Practice and published by the World Bank, it is written by development practitioners – Paul J. Gertler, Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch – who have done tens of evaluations, who understand the difficulties of conducting good impact evaluations, and know ways of overcoming these difficulties. It is a “how to” book that is presented chronologically and skillfully holds your hand taking you step-by-step through a journey of evaluation, answering all your ‘unasked’ questions, while imparting knowledge. This new book is available to everyone for free here.

Impact Evaluation in Practice is written in a conversational style, making it an easy read for practitioners and for students of evaluation. The fact that it is written by practitioners makes the examples very relevant and credible. The authors depict evaluation as an applied science whose results inform policy, and like any science, they show that evaluations must be methodical and systematic evidence-based and open to scrutiny if the data is going to inform decisions.

This book is suitable for all those who want to manage for results. It clearly shows one how to include impact evaluations in the design stages of development projects, and the key considerations to make. As managers of consultants that evaluate development projects, you will be ahead of your ordinary consultant if you have read this book.

The part on the use of evaluation to inform policy is critical and is most important for all of us who embrace MfDR, given that evaluations are the basis for learning what works, what does not work and how we use the evidence to formulate policies and to inform the development of new projects. Poor or weak evaluations therefore can bring with them misleading evidence. Being methodical in an evaluation means you can easily question and verify a finding and you can also support a decision using data-evidence.

Caution: This book is about the ideal. Our realities are sometimes far from ideals, so those using the book to guide evaluations should be mindful of the fact. For instance, many projects that are evaluated can have logical frameworks – logframes – which are illogical with “unsmart” results statements and indicators, as well as activities and outputs that have remained frozen for 5 years (meaning strategies were not adapted as the project context changed); thus rendering the monitoring of data possibly less credible. That becomes a great challenge for an evaluator. Another challenge in practice is that the evaluator is often given 25 days and at most 30 days to evaluate and report of findings. With limited time, it is sometimes not possible to be as systematic and methodical as required.

All in all, this is an excellent book with invaluable insights, and I unreservedly recommend it to all those in development who design evaluations, those that evaluate, teach evaluation, supervise evaluations and those that are students of evaluation. The book clearly makes the distinction between monitoring and evaluation, and the relationship between the two.

Rosa Muraguri-Mwololo, Kenya

Program Review Committee Secretary at UN-Habitat

Kenya CoP on MfDR, Founding Member

Former AfCoP Core Management Team Member