The End of the (Research) World As We Know It? Understanding and Coping With Declining Response Rates to Mail Surveys

Abstract

Mail surveys have long been a staple of social science research. Properly conducted, they can gather representative data about a population that provides important generalizations about that population. High response rates are one crucial element of this capacity to make such generalizations. Response rates to mail surveys–especially those targeting general populations–have declined substantially over the years, even when employing standard “best practices” such as carefully crafted and pretested instruments, good explanations of research purpose, and multiple contacts. Our research group has a unique capacity to speak to declining response rates, having implemented mail surveys on natural resource-related topics–using relatively similar methods–for more than 45 years. We present the results of a longitudinal analysis of response rates to 191 surveys we conducted between 1971 and 2017 (response rates and methodological specifics carefully recorded over this time), documenting the changes in response rates over the years, and the factors associated with these changes. Variables assessed include topic saliency, the nature of the sample, timing, length and complexity of the questionnaire, and others. We find that year of implementation is a strong predictor of response rate, net of these other factors. We use these findings as a base from which to engage a larger discussion about strategies that social science researchers might use, both in terms of continued modifications to mail survey approaches and in moving towards other research methods.