Why We Need to Rethink Measurement Invariance: The Role of Measurement Invariance for Cross-Cultural Research

Abstract

Claims about human behavior have been hampered by limited availability of comparable data across cultures. Invariance testing has been proposed to address questions about the comparability of data, yet statistical methods have been challenged on various grounds across psychology and related fields. We highlight how current debates confuse distinct issues and fail to consider the role of data within science. We aim to overcome the impasse by a) summarizing various criticisms, b) distinguishing five mapping processes that occur during a typical research project and c) exploring how thinking about invariance as mapping can move current discussions forward. Specifically, we differentiate 1) mapping ideas to theoretical constructs and concepts, 2) mapping constructs to stimuli, 3) mapping participants’ responses to stimuli onto numerical representations, 4) testing internal relations of stimuli responses (the typical focus of statistical invariance testing) and 5) mapping empirical observations back to theoretical statements. We treat invariance testing as a theory-guided process that offers important insights about instruments, construct validity and psychological theories throughout the research process that are currently missed by focusing on only the statistical details. Psychological claims that are valid for all humans depend on questions of invariance in the broad sense that we outline here.