Relationships between ignoring instructions and response bias when completing questionnaires

Abstract

Certain participants are insincere, or careless when they respond to questionnaires. To identify such participants, we included three items in a questionnaire that instructed participants to choose a particular response category. Nurses (N = 1,000) responded to this questionnaire in a Web survey. One-hundred-twenty participants failed to follow the instructions for at least one item (non-followers). Analyzing their responses indicated the following: (a) non-followers were more likely to give identical, or midpoint responses; (b) the correlations between their responses to regular and reversed items were low or positive, and their responses to scales containing reversed items tended to show lower internal consistency; and finally, (c) the mean scores of non-followers were closer to the midpoint of the scale, regardless of whether the scale included reversed items. One reason that including reversed items lead to lower internal consistency could be because participants occasionally missed responding to these items. However, the results suggested that non-followers were not diligent in responding to regular items, and merely deleting reversed items from scales will be insufficient to ensure accurate results.