Coding Issues of Open-Ended Questions in a Cross-Cultural Context

Abstract

Although cross-cultural surveys increasingly use open-ended questions to obtain detailed information on respondents’ attitudes, the issue of coding quality is rarely addressed. These questions are always challenging but even more so in multilingual, cross-cultural research contexts as the different survey languages make response coding more difficult and costly. In this paper, we examine coding issues of open-ended questions and the impact of translation on coding results by comparing codings of translated responses (two-step approach with translation and coding) with codings of the same responses in the original languages (one-step approach using bilingual coders). We draw on data from the project CICOM, specifically respondents’ answers in English and Spanish to open-ended questions about the meaning of left and right. Our goal is to determine whether the coding approach makes a difference to data quality and to identify error sources in the process. Positive news is that both coding approaches resulted in good quality data. We identify several error sources related, first, to respondents’ short answers; second, to the translation process; and third, to the coding process. The response context and the cultural background of translators and coders appear to be important.