Measuring consumer sensitivity to audio advertising: a long-run field experiment on Pandora internet radio

Abstract

A randomized experiment with almost 35 million Pandora listeners enables us to measure the sensitivity of consumers to advertising, an important topic of study in the era of ad-supported digital content provision. The experiment randomized listeners into nine treatment groups, each of which received a different level of audio advertising interrupting their music listening, with the highest treatment group receiving more than twice as many ads as the lowest treatment group. By maintaining consistent treatment assignment for 21 months, we measure long-run demand effects and find ad-load sensitivity three times greater than what we would have obtained from a month-long experiment. We show the negative impact on the number of hours listened, days listened, and probability of listening at all in the final month. Using an experimental design that separately varies the number of commercial interruptions per hour and the number of ads per commercial interruption, we find that listeners primarily respond to the total number of ads per hour, with a slight preference for more frequent but shorter ad breaks. Lastly, we find that increased ad load led to an increase in the number of paid ad-free subscriptions to Pandora. Our analysis demonstrates that relying on observational data often leads to biased or even directionally incorrect estimates of these effects, highlighting the value of experimental variation.