Evaluating the effectiveness of multiple-answer formats in volunteer panel Web surveys

Abstract

This study examined the inhibiting factors of satisficing response on multiple-answer format (MA) in volunteer panel Web surveys. Data were based on two Web surveys in the Tokyo metropolitan area (one of 2,257 Japanese adults in March 2014, and the other of 519 Japanese adults in October 2017). The results revealed the following: (a) respondents endorsed fewer options and took less time to answer MA than forced choice format (FC); (b) an acquiescence tendency was not found for almost all the items in FC; and (c) satisficing responses were not inhibited by the number or properties of items. Results (a) and (b) suggest that MA may encourage satisficing response strategies. Furthermore, these tendencies can be seen not only in attitudinal (judgment-type) questions, but also behavioral (recall-type) questions. These results indicate the strength of satisficing response behaviors, so the use of FC is recommended in volunteer panel Web surveys.