Best practices for using natural experiments to evaluate retail food and beverage policies and interventions

Abstract

Policy and programmatic change in the food retail setting, including excise taxes on beverages with added-caloric sweeteners, new supermarkets in food deserts, and voluntary corporate pledges, often require the use of natural experimental evaluation for impact evaluation when randomized controlled trials are not possible. Although natural experimental studies in the food retail setting provide important opportunities to test how nonrandomized interventions affect behavioral and health outcomes, researchers face several key challenges to maintaining strong internal and external validity when conducting these studies. Broadly, these challenges include 1) study design and analysis; 2) selection of participants, selection of measures, and obtainment of data; and 3) real-world considerations. This article addresses these challenges and different approaches to meeting them. Case studies are used to illustrate these approaches and to highlight advantages and disadvantages of each approach. If the trade-offs required to address these challenges are carefully considered, thoughtful natural experimental evaluations can minimize bias and provide critical information about the impacts of food retail interventions to a variety of stakeholders, including the affected population, policymakers, and food retailers.