Impact Evaluations and Development

Abstract

The Guidance document was written by and represents the views of the authors. Given the fact that perspectives on the definition, scope, and appropriate methods of impact evaluation differ widely among practitioners and other stakeholders, the document should not be taken to represent the agreed positions of all of the individual NONIE members. Why promote impact evaluations? For develop- ment practitioners, impact evaluations play a key role in the drive for better evidence on results and development effectiveness. They are particularly well suited to answer important questions about whether development interventions do or do not work, whether they make a difference, and how cost-effective they are. Consequently, they can help ensure that scarce resources are allocated where they can have the most developmental impact. Although there is debate within the profession about the precise definition of impact evalua- tion, NONIEs use of the term proceeds from its adoption of the Development Assistance Committee of the Organisation for Economic Co-operation and Development (DAC) definition of impact, as the positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.2 Adopting the DAC definition of impact leads to a focus on two underlying premises for impact evaluations: Attribution: The words effects produced by in the DAC definition imply an approach to impact evaluation that is about attributing im- pacts to interventions, rather thanjust assess- ing what happened. Counterfactual: It follows that in most contexts, knowledge about the impacts produced by an intervention requires an attempt to gauge what would have occurred in the absence of the intervention and a comparison with what has occurred with the intervention implemented. These two premises do not, however, lead to a determination of a set of analytical methods that is above all others in all situations. In fact, this Guidance note underlines that No single method is best for addressing the variety of questions and aspects that might be part of impact evaluations. However, depending on the specific questions or objectives of a given impact evaluation, some methods have a comparative advan- tage over others in analyzing a particular ques- tion or objective. Particular methods or perspectives comple- ment each other in providing a more complete picture of impact. The document is structured around nine key issues that provide guidance on conceptualizing, designing, and implementing an impact evalua- tion: Methodological guidance: 1. Identify the type and scope of the interven- tion. These two premises do not, however, lead to a determination of a set of analytical methods that is above all others in all situations. In fact, this Guidance note underlines that No single method is best for addressing the variety of questions and aspects that might be part of impact evaluations. However, depending on the specific questions or objectives of a given impact evaluation, some methods have a comparative advan- tage over others in analyzing a particular ques- tion or objective. Particular methods or perspectives comple- ment each other in providing a more complete picture of impact. The document is structured around nine key issues that provide guidance on conceptualizing, designing, and implementing an impact evalua- tion: Methodological guidance: 1. Identify the type and scope of the interven- tion.