Impact of survey length and compensation on validity, reliability, and sample characteristics for Ultrashort-, Short-, and Long-Research Participant Perception Surveys

Abstract

Introduction The validated long Research Participant Perception Survey (RPPS-Long) elicits valuable data at modest response rates. Methods To address this limitation, we developed shorter RPPS-Ultrashort and RPPS-Short versions, fielded them with the RPPS-Long to a random sample of a national research volunteer registry, and assessed response and completion rates, test/retest reliability, and demographics. Results In total, 2228 eligible registry members received survey links. Response rates were 64% (RPPS-Ultrashort), 63% (RPPS-Short), and 51% (RPPS-Long), respectively (p<0.001). Completion rates were 63%, 54%, and 37%, respectively (p<0.001). All surveys were reliable with Cronbach α=0.81, 0.84, and 0.87, respectively. Retest reliability was highest for RPPS-short (κ=0.85). Provision of compensation increased RPPS-short completion rate from 54% to 71% (p<0.001). Compensated respondents were younger (p<0.001), with greater minority representation (p=0.03). Conclusions Shorter surveys were reliable and produced higher response and completion rates then long surveys. Compensation further increased completion rates and shifted sample age and race profiles.