Examining the relationship between nonresponse propensity and data quality in two national household surveys

Abstract

Important theoretical questions in survey research over the past 50 years have been: How does bringing in late or reluctant respondents affect total survey error? Does the effort and expense of obtaining interviews from difficult-to-contact or reluctant respondents significantly decrease the nonresponse error of survey estimates? Or do these late respondents introduce enough measurement error to offset any reductions in nonresponse bias? This study attempts to address these questions by examining nonresponse and data quality in two national household surveys: the Current Population Survey (CPS) and the American Time Use Survey (ATUS). Response propensity models were developed for each survey, and data quality in each survey was assessed by a variety of indirect indicators of response error, for example, item-missing-data rates, round value reports, and interview-reinterview response inconsistencies. The principal analyses investigated the relationship between response propensity and the data-quality indicators in each survey, and examined the effects of potential common causal factors when there was evidence of covariation. Although the strength of the relationship varied by indicator and survey, data quality decreased for some indicators as the probability of nonresponse increased. Therefore, the direct implication for survey managers is that efforts to reduce nonresponse can lead to poorer-quality data. Moreover, these effects remain even after attempts to control for potential common causal factors. © The Author 2011.