Impact of Question Type and Question Order on Tobacco Prevalence Estimates in US Young Adults: A Randomized Experiment

Abstract

The purpose of this study was to identify the effects of question type, order, and inclusion of product images on tobacco use estimates in a national sample of young adults. Participants aged 18-34 years (N = 4,100) in the Truth Initiative Young Adult Cohort Study (2016) were randomized to one of five question types assessing ever and past 30-day use of tobacco products: (1) "select all that apply" list (checklist, CL); (2) breakout items for each product (B); (3) breakout + images (B + I); (4) CL and B; and (5) CL and B + I. The order of question type was randomly assigned in groups 4 and 5. Bivariate analyses estimated product-specific prevalence by question type/order. Ever cigarette and cigar use prevalence was higher and ever e-cigarette use was lower in B and B + I than in CL. Ever hookah use was higher in B + I than in CL. Past 30-day e-cigarette use was 8.3% higher and past 30-day smokeless use was 13.0% higher in B + I than in CL. In groups 4 and 5, higher prevalence of ever cigarette, cigar, hookah, and past 30-day smokeless use was observed when B was presented first. Question type, order, and inclusion of images affect prevalence estimates of tobacco use. Implications: This study identifies the effects of question type, order, and inclusion of product images on tobacco use estimates in a national sample of young adults. Ever use and past 30-day use prevalence estimates of specific products were affected by respondents answering breakout items or breakout items with images compared with respondents answering a checklist of items in the survey. Current surveys that include a "select all that apply" list format may be underestimating ever and past 30-day prevalence estimates of tobacco products.