Digital questionnaire response time (DQRT): A ubiquitous and low-cost digital assay of cognitive processing speed

Abstract

Digital survey tools have all but replaced paper and pen in the psychological sciences, and consequently new forms of potentially useful research paradata are now routinely gathered. A particularly common byproduct of research is questionnaire timestamps, which some have suggested can be used as a measure of cognitive function. Here, we conducted a comprehensive validation of this measure, which we call the “digital questionnaire response time,” or “DQRT.” Using data from N = 2,977 users of a smartphone app, we first ran a data-driven bootstrapping approach to examine how best to quantify DQRT. DQRT was slower in older adults (r = 0.26) and in those with lower educational attainment and socioeconomic status. Testing the association between DQRT and working memory (range r = 0.11–0.14), model-based planning (range r = 0.03–0.06), and processing speed (range r = 0.29–0.39) across cross-sectional and longitudinal subsamples, we found support for a cognitive characterization of DQRT as a measure of cognitive processing speed. DQRT was more strongly correlated with nine out of 13 lifestyle and health factors, and four out of nine mental health factors than a task-based measure of processing speed. DQRT showed good test–retest reliability, and associations between DQRT and task-based processing speed were higher within individuals (r = 0.35) than between individuals (r = 0.25). Finally, we highlight substantial, but addressable, potential confounds inherent in the measure. We conclude that DQRT has important limitations, but overall can serve as a valid and reliable index of cognitive processing speed that can be gathered at unprecedented scale, unobtrusively, and repeatedly, during a variety of real-world digital behaviors.