Assessing the effect of questionnaire design on unit and item-nonresponse: evidence from an online experiment

Abstract

Questionnaire design choices, such as handling ‘do not know’ answers or using check all that apply questions may impact the level of item- and unit nonresponse. We used experimental data from an online panel and the questions of ESS to examine how applying forced answering, offering DK/NA options and specific question formats (CATA vs. forced choice format) impact the level of nonresponse and how respondent characteristics correlate with these effects. The results suggest that neither forcing respondents nor offering an explicit NR option is advisable, as these options strongly increase nonresponse. Rather a skipping-allowed design is a more reasonable choice. Those who were interested in the topic of the survey were far less sensitive to NR handling. Developing adaptive questionnaire designs based on topic interest can be a promising direction for future research.