Is that still the same? Has that changed? On the accuracy of measuring change with dependent interviewing

Abstract

Measurement and analysis of change is one of the primary reasons to conduct panel surveys, but studies have shown that estimates of change from panel surveys can be subject to measurement error, most commonly overreporting of change. For this reason, many panel surveys use a technique called proactive dependent interviewing, which reminds respondents of their answer in the previous wave, and has been shown to reduce the capturing of spurious change. However, so far very little guidance exists in the literature on how such questions should be worded. Here, we use data from three experimental studies to examine question wording effects with proactive dependent interviewing. Because we link data from one of the surveys to administrative records, we can examine not only different levels of change by format, but the accuracy of the change reports as well. Our results show that how questions about current status are worded affects the reporting of change. The overall results, including comparisons with administrative records, suggest that reminding respondents of their previous answer and then asking, “Is that still the case?” produces the most accurate data on change and stability experienced by respondents.