Testing the effectiveness of lottery incentives in online experiments

Abstract

This article investigates the effectiveness of lottery incentive schemes for eliciting consumer valuations in large-scale online experiments. We implement a fully incentivized condition within a geographically dispersed sample of consumers in which bids for a Criollo steak elicited by a Becker-DeGroot-Marschak mechanism are realized with certainty and the products are priority shipped in dry-ice coolers. The fully incentivized condition is compared to between-subject random incentivized schemes, in which only a fraction of subjects realize their choices. We tested two treatments with a 10% probability framed as a percentage or an absolute number of subjects, one treatment with a 1% probability, and a purely hypothetical reference condition. The results reveal that between-subject random incentivized schemes with 10% and 1% payment probabilities are effective in eliciting valuations that are statistically indistinguishable from the fully incentivized scheme. In addition to finding insignificant statistical differences between 10% and 1% and the fully incentivized scheme, all incentivized conditions mitigate hypothetical bias, resulting in lower product valuations than the purely hypothetical condition. We contribute a novel methodological framework for conducting large-scale experiments with geographically diverse and representative subjects, increasing the external validity and producing reliable valuations while significantly reducing financial and logistic constraints.