Measuring the Hard-to-Measure in Development: Dimensions, Measurement Challenges, and Responses

Abstract

Development is a multi-dimensional, imprecise concept. Initiatives that aim to improve development attempt to address entrenched economic and social issues, increasingly through multi-component programmes, involve diverse sets of stakeholders pursuing different, sometimes competing interests, and must adapt to shifting contexts. They operate under conditions of uncertainty and complexity. Each of these factors poses challenges for measurement validity and reliability. Based on common challenges that arose through development initiatives across a range of contexts, this paper explores four hard-to-measure dimensions of development: abstract, multi-dimensional concepts, processes, and issues; challenging settings where there are unpredictable, sudden or frequent shifts in the environment; multiple, uncertain pathways of change; and multi-layer implementing structures such as cross-sector partnerships and regional/national/subnational initiatives. These dimensions pose specific difficulties related to what is measured and where, the assessment of how and why changes took place, and who or what is the unit of analysis. They affect the credibility of evidence, a foundational element of evidence-informed decision-making. In addition to technical, methodological aspects, relational and political factors also have implications for measurement, even if the issue area, setting, pathway of change, and implementing structure are not inherently difficult.