Are Mechanical Turk worker samples representative of health status and health behaviors in the U.S.?

Abstract

Introduction Amazon’s Mechanical Turk (MTurk) is frequently used to administer health-related surveys and experiments at a low cost, but little is known about its representativeness with regards to health status and behaviors. Methods A cross-sectional survey comprised of questions from the nationally-representative 2014 Behavioral Risk Factor Surveillance System (BRFSS) and 2014 National Health and Nutrition Examination Survey (NHANES) was administered to 591 MTurk workers and 393 masters in 2016. Health status (asthma, depression, BMI, and general health), health behaviors (influenza vaccination, health insurance, smoking, and physical activity), and demographic characteristics of the two MTurk populations (workers and masters) were compared to each other and, using Poisson regression, to a nationally-representative BRFSS and NHANES samples. Results Workers and master demographics were similar. MTurk users were more likely to be aged under 50 years compared to the national sample (86% vs. 55%) and more likely to complete a college degree than the national sample (50% vs. 26%). Adjusting for covariates, MTurk users were less likely to be vaccinated for influenza, to smoke, to have asthma, to self-report being in excellent or very good health, to exercise, and have health insurance but over twice as likely to screen positive for depression relative to a national sample. Results were fairly consistent among different age groups. Conclusions MTurk workers are not a generalizable population with regards to health status and behaviors; deviations did not follow a trend. Appropriate health-related uses for MTurk and ways to improve upon the generalizability of MTurk health studies are proposed.