Comparing Item Nonresponse across Different Delivery Modes in General Population Surveys

Abstract

In 2006, we began a multi-year effort to evaluate the possibility of transitioning from a regularly conducted telephone survey of general public satisfaction with Oregon transportation services to either a mail-only or Web-plus-mail mixed-mode design. Two primary objectives were: 1) to identify ways to counter the trend toward lower telephone survey response rates and 2) to evaluate the impact of alternative designs on data quality. Three experiments were conducted for which unit response rates analyses have been reported elsewhere (Lesser and Newton, 2007; Lesser et al., 2011a; Lesser et al., 2011b). In this study, we consider whether switching from telephone to mail or a combination of mail and Web will have a deleterious effect on item nonresponse, as previous research has suggested (e.g., de Leeuw et al., 2003). Thus, our purpose is to examine overall item nonresponse rates for each of the individual modes and mode combinations. Implications for replacing telephone by these alternatives are also discussed. Methods The experiments were conducted by the Oregon State University Survey Research Center (OSU-SRC) in three separate years: 2006, 2008, and 2010. The questionnaire was similar each year, asking random samples of Oregon households their opinions on the quality of various transportation modes within the state. The paper version of the questionnaire in all studies across all years consisted of 12 pages (including a cover page), and requested answers to 95 to 109 questions over the three years. The Web and telephone versions of