A reply to “Do surveys accurately report voters over 80 years old?”: testing for bias in probability-based surveys of private households

Abstract

This article is a reply and extension to a paper published by Jean-Yves Dormagen and Laura Michel in this journal in early 2018. In their paper “Do surveys accurately report voters over 80 years old?”, the authors examined the deviations of aggregated self-reported turnout in three French electoral surveys from the factual turnout in the French presidential elections of 2002, 2007 and 2012. The tested election studies used a quota sampling method, which suffers most likely from selection bias. Indeed, the authors found a clear underrepresentation of the oldest population older than 85 years in all three studies and a strong overestimation of voting in this age cohort. In this article, I replicate their analysis with probability-based surveys and test whether the bias in turnout of the elderly also occurs in randomly drawn samples. For this purpose, I analysed data of three cross sections of the European Social Survey (ESS-1, ESS-4 and ESS-6). My results show that the probability-based surveys are indeed closer to true values derived from large-scale administrative data. The bias in turnout in the ESS data is nevertheless still stronger in the oldest age cohorts compared to all the other age cohorts. This could be the cause of a higher nonresponse among the elderly. Moreover, the ESS does not cover the institutionalised population. In France, more than 13% of the population older than 80 years are institutionalised and live in retirement and nursing homes. I assume that this part of the population has a lower probability to vote than the elderly living in private households. Their exclusion from the ESS therefore increases the distortion of self-reported voting behaviour compared to the factual turnout of old-aged residents.