Examining Social Desirability Bias in Online and Offline Surveys

Abstract

This paper examines whether web-based survey responses differ from traditional mail-based questionnaire responses when examining attitudes towards sensitive issues in a mixed-mode survey. Our motivation for the study comes from social desirability bias, which is a generally discussed theme in social surveys. The data are derived from the Finnish section of the International Social Survey Program (ISSP) 2013 (n = 1,243), which applied both self-conducted mail survey and web survey data collection techniques. In the analysis, we utilize items concerning attitudes towards immigrants. We found that mail-questionnaire respondents tend to express more negative attitudes towards immigration than the web-questionnaire group. This is especially true when analyzing the survey items, which use negative connotations in the question formulation. The results also indicated that socio-demographic background had a significant impact on responses, but these factors did not explain the total variation between the two response groups. We discuss our findings in light of the popularity of web-surveys and their increased use in social sciences. We conclude our paper with a notion that the mixed-mode survey is a reliable method of data collection, especially after controlling for relevant background variables, and their interactions between the alternative response modes.