Measurement requires compromises: the case of economic inequality

Abstract

We examine considerations that enter into design and evaluation of measures in social science, categorizing them into four drivers: epistemic, ethical, pragmatic, and metrological. We call them drivers to highlight their role in guiding researchers’ decisions without determining them. Through an analysis of the World Inequality Report 2022, we reveal tensions among these drivers, illustrating the complex interplay between the various demands a measure must satisfy. Our analysis highlights the need for case-by-case compromises to address these tensions, as optimizing one driver often comes at the expense of another. We explore the extent to which these compromises shape measurement practice and the principles that guide researchers in balancing them. While existing literature on measurement assumes that tensions can be resolved with good practice and use, we argue that developing a good measure requires balancing multiple demands, recognising that it might be impossible to meet all of them simultaneously.