Scientific Surveys Based on Incomplete Sampling Frames and High Rates of Nonresponse

Abstract

Traditional methods of survey research that rely on Neyman’s probability-based sampling paradigm are grounded in a number of fundamental assumptions that are becoming exceedingly difficult to attain in today’s survey research environment. On the one hand, common methods of sampling are subject to coverage issues that may not be fully ameliorated through post-survey weighting adjustments. On the other, response rates continue to deteriorate for all surveys, even when resource-intensive refusal conversion strategies are employed. Add in the growing need for cost containments and it is no wonder why alternative sampling methods are gaining popularity. The authors will review a number of practices that are currently used for developing inferences from samples that do not fully adhere to the statistical machinery that is currently available for probability-based sample surveys. Moreover, a robust weighting methodology will be introduced that can reduce the inherent biases associated with non-probability samples, as well as probability-based sample surveys that suffer from incomplete frames and high rates of nonresponse. The efficacy of the proposed methodology is assessed in light of comparisons of survey estimates to external benchmarks, relying on parallel surveys that were conducted in two states using both probability-based and non-probability samples.