Do Mail and Internet Surveys Produce Different Item Nonresponse Rates? An Experiment Using Random Mode Assignment

Abstract

A significant limitation of most data quality comparisons between Web and mail survey responses is that the individuals who choose to respond to one survey mode have different characteristics than those who select the alternative mode. As suggested by other papers in this issue of Survey Practice, such differences in, for example, Internet access, education, and income, may contribute to mode differences in item nonresponse. This makes it difficult to isolate how mode itself affects this aspect of survey quality. The surveys of undergraduate students analyzed herein differ from general public surveys in this regard, inasmuch as all students in this fairly homogenous (for education and age) population have Internet access and are accessible by both postal mail and e-mail. Since college students are relatively highly Internet-literate and course work requires nearly daily use of the Internet, it is possible to randomly assign students to Web-only and mail-only treatment groups. In this paper, we determine if mail and Web item nonresponse rates are similar when randomly assigning individuals to a response mode. Additionally, we examine patterns of item nonresponse for different types of question formats. Data and Methods We examined results from two surveys of randomly selected undergraduate students at Washington State University (WSU). The surveys were conducted in the spring and fall of 2009, respectively. The first study focused on students' general satisfaction with WSU and their educational experiences. The second study primarily asked about the effects of the economic downturn and university