Criterion validity of five open-source app-based cognitive and sensory tasks in an Australian adult life course sample aged 18 to 82: Labs without walls

Abstract

With recent technical advances, many cognitive and sensory tasks have been adapted for smartphone testing. This study aimed to assess the criterion validity of a subset of self-administered, open-source app-based cognitive and sensory tasks by comparing test performance to lab-based alternatives. An in-person baseline was completed by 43 participants (aged 21 to 82) from the larger Labs without Walls project (Brady et al., 2023) to compare the self-administered, app-based tasks with researcher-administered equivalents. 4 preset tasks sourced from Apple’s ResearchKit (Spatial Memory, Trail Making Test, Stroop Test, and dBHL Tone Audiometry) and 1 custom-built task (Ishihara Color Deficiency Test) were compared. All tasks except the Spatial Memory task demonstrated high comparability to the researcher-administered version. Specifically, the Trail Making Tests were strongly correlated (.77 and.78 for parts A and B, respectively), Stroop correlations ranged from.77 to.89 and the Ishihara tasks were moderately correlated (r =.69). ICCs for the Audiometry task ranged from.56 to.96 (Moderate to Excellent) with 83% sensitivity and 100% specificity. Bland–Altman plots revealed a mean bias between -5.35 to 9.67 dB for each ear and frequency with an overall bias of 3.02 and 1.98 for the left and right ears, respectively, within the minimum testing interval. Furthermore, all app-based tasks were significantly correlated with age. These results offer preliminary evidence of the validity of four open-source cognitive and sensory tasks with implications for effective remote testing in non-lab settings.