Comparative Research is Harder Than We Thought: Regional Differences in Experts' Understanding of Electoral Integrity Questions

Abstract

Expert evaluations about countries form the backbone of comparative political research. It is reasonable to assume that such respondents, no matter the region they specialize in, will have a comparable understanding of the phenomena tapped by expert surveys. This is necessary to get results that can be compared across countries, which is the fundamental goal of these measurement activities. We empirically test this assumption using measurement invariance techniques which have not been applied to expert surveys before. Used most often to test the cross-cultural validity and translation effects of public opinion scales, the measurement invariance tests evaluate the comparability of scale items across any groups. We apply them to the Perceptions of Electoral Integrity (PEI) dataset. Our findings suggest that cross-regional comparability fails for all eleven dimensions identified in PEI. Results indicate which items remain comparable, at least across most regions, and point to the need of more rigorous procedures to develop expert survey questions.