Difficulties of Measuring Culture in Privacy Studies

Abstract

This paper addresses inconsistencies that exist in the measurement instruments HCI researchers use in cross-cultural studies. We study some commonly used measurement instruments that capture cultural dimensions at an individual level and conduct "measurement invariance tests,"which test whether the questions comprising a construct have similar characteristics across different groups (e.g., countries). We find that these cultural dimensions are, to some extent, non-invariant, making statistical comparisons between countries problematic. Furthermore, we study the (non)invariance of the causal relationship between these cultural dimensions and privacy-related constructs, e.g., privacy concern and the amount of information users share on social media. Our results suggest that in several instances, these cultural dimensions have a different effect on privacy-related constructs per country. This severely reduces their usefulness for developing cross-cultural arguments in cross-country studies. We discuss the value of conducting measurement and causal non-invariance tests and urge scholars to develop more robust means of measuring culture.