The effects of task difficulty and conversational cueing on answer formatting problems in surveys

Abstract

In this chapter we discuss three types of formatting problems: (i) mismatch answers, (i.e. answers that deviate from prescribed categories), (ii) reports, (i.e. potentially relevant information from which an answer may be derived), and (iii) qualified answers (i.e. answers that are accompanied by a qualification). In questionnaire design, we can influence the occurrence of these formatting problems with two essentially opposing strategies. With conversational cueing, we adjust the survey design to the respondent’s conversational tendencies by using colloquial response alternatives. With reduced task difficulty, we force respondents to use the “language of surveys” but try to decrease formatting problems by using question wording that helps them construct their answer according to the required format. The results of our experimental study show that reduced task difficulty did not reduce mismatch answers, but it did decrease the number of reports. Using colloquial response alternatives significantly reduced all three types of formatting problems.