Supporting people with dementia- Understanding their interactions with Mixed Reality Technologies

Abstract

Emerging technologies such as Mixed Reality Technologies (MRTs) could offer possibilities to support People with Dementia (PWD). This study examined interactions of PWD with two MRTs - HoloLens and Osmo. Ten participants (MoCA = 18 to 23, Age= 63 to 88 years) played a game of Tangram on Osmo. Six of these participants played Young Conker on HoloLens. The study found that PWD used gestural actions in the physical world more correctly than speech. Audio prompts in human voice were more correctly perceived than visual prompts. Physical affordances, embodied actions and familiarity to verbal instructions contribute to this success. Visual prompts such as text present promising opportunities to complement with audio prompts. Interaction with MRTs require prompts to direct PWD towards physical or virtual worlds. The research outcomes are significant as the focus on interactions of PWD could open up avenues for further research on actions and perceptions with emerging technologies.