Conversational artificial intelligence and affective social robot for monitoring health and well-being of people with dementia

Abstract

BACKGROUND: Social robots are anthropomorphised platforms developed to interact with humans, using natural language, offering an accessible and intuitive interface suited to diverse cognitive abilities. Social robots can be used to support people with dementia (PwD) and carers in their homes managing medication, hydration, appointments, and evaluating mood, wellbeing, and potentially cognitive decline. Such robots have potential to reduce care burden and prolong independent living, yet translation into PwD use remains insignificant. METHOD: We have developed two social robots - a conversational robot and a digital social robot for mobile devices capable of communicating through natural language (powered by Amazon Alexa) and facial expressions that ask PwD daily questions about their health and wellbeing and also provide digital assistant functionality. We record data comprising of PwD's responses to daily questions, audio speech and text of conversations with Alexa to automatically monitor their health and wellbeing using machine learning. We followed user-centric development processes by conducting focus groups with 13 carers, 2 PwD and 5 clinicians to iterate the design. We are testing social robot with 3 PwD in their homes for ten weeks. RESULT: We received positive feedback on social robot from focus group participants. Ease of use, low maintenance, accessibility, assistance with medication, supporting with health and wellbeing were identified as the key opportunities for social robots. Based on responses to a daily questionnaire, our robots generate a report detailing PwD wellbeing that is automatically sent via email to family members or carers. This information is also stored systematically in a database that can help clinicians monitor their patients remotely. We use natural language processing to analyse conversations and identify topics of interest to PwD such that robot behaviour could be adapted. We process speech using signal processing and machine learning classifiers to identify emotions and mood. CONCLUSION: Affective social robots with voice and facial expressions have a great potential for enhancing dementia care at home. Combining responses to daily questions, topics of conversations and emotions from speech analysis can help assess daily status of PwD automatically and monitor their health and well-being over time remotely.