Does a probability-based household panel benefit from assignment to postal response as an alternative to internet-only

Abstract

A potential limitation of web-only panels of the general public, even when households are selected using probability methods, is that only about 70 percent of U.S. households have members with Internet access. In addition, some members of Internet-connected households may be unable or unwilling to participate over the web. The Gallup Panel uses both web and mail modes to survey respondents and in 2006 included approximately 50,000 households selected by random-digit dialing. Frequent Internet users were assigned to respond by the web, while others were assigned to participate by mail using a paper questionnaire with a similar visual layout to the web. We use several approaches to determine whether or not the mail option adds value to the results in an otherwise Internet panel and organize our analyses around answering a series of questions. First, does the use of mail allow different types of people to be included Second, do mail and web respondents give different answers to the same questions Third, does weighting on and controlling for demographics eliminate any differences in responses from mail and web respondents and indicate that mail is not needed Finally, do differences exist when responses are collected using an independent mode In general, the answers to these questions suggest that use of mail adds value to the panel results and improves the overall accuracy of survey results.