A SLAM-based augmented reality app for the assessment of spatial short-term memory using visual and auditory stimuli

Abstract

A SLAM-based Augmented Reality (AR) app has been designed, developed, and validated to assess spatial short-term memory. Our app can be used with visual and auditory stimuli and can run on mobile devices. It can be used in any indoor environment. The anchors and data of the app are persistently stored in the cloud. As an authoring tool, the type of stimulus, its number, and specific positions in the real environment can be customized for each session. A study involving 48 participants was carried out to analyze the performance outcomes comparing the location and remembering of stimuli in a real environment using visual versus auditory stimuli. The number of objects placed correctly was similar for the two different stimuli used. However, the group that used the auditory stimulus spent significantly more time completing the task and required significantly more attempts. The performance outcomes were independent of age and gender. For the auditory stimuli, correlations among all of the variables of the AR app and the variables of two other tasks (object-recall and map-pointing) were found. We also found that the greater the number of correctly placed auditory stimuli, the greater the perceived competence and the less mental effort required. The greater the number of errors, the less the perceived competence. Finally, the auditory stimuli are valid stimuli that may benefit the assessment of the memorization of spatial-auditory associations, but the memorization of spatial-visual associations is dominant, as our results suggest.