Eliciting subjective probabilities in internet surveys

Abstract

Individuals' subjective expectations are important in explaining heterogeneity in individual choices, but their elicitation poses some challenges, in particular when one is interested in the subjective probability distribution of an individual. We have developed an innovative visual representation for Internet surveys that has some advantages over previously used formats. In this paper we present our findings from testing this visual representation in the context of individuals' Social Security expectations. Respondents are asked to allocate a total of 20 balls across seven bins to express what they believe the chances to be that their future Social Security benefits would fall into any one of those bins. Our data come from the Internet survey of respondents to the Health and Retirement Study, a representative survey of the U.S. population aged 51 and older. To contrast the results from the visual format with a previously used format, we divided the sample into two random groups and administered both, the visual format and the more standard percent chance format. Our findings suggest that the main advantage of the visual format is that it generates usable answers for virtually all respondents in the sample while in the percent chance format a significant fraction (about 20 percent) of responses is lost due to inconsistencies. Across various other dimensions, the visual format performs similarly to the percent chance format, leading us to conclude that the bins-and-balls format is a viable alternative that leads to more complete data.