Emotion expression in a socially assistive robot for persons with Parkinson's disease

Abstract

Emotions are crucial for human social interactions and thus people communicate emotions through a variety of modalities: kinesthetic (through facial expressions, body posture and gestures), auditory (the acoustic features of speech) and semantic (the content of what they say). Sometimes however, communication channels for certain modalities can be unavailable (e.g., in the case of texting), and sometimes they can be compromised, due to a disorder such as Parkinson's disease (PD) that may affect facial, gestural and speech expressions of emotions. To address this, we developed a prototype for an emoting robot that can detect emotions in one modality, specifically in the content of speech, and then express them in another modality, specifically through gestures. The system consists of two components: detection and expression of emotions. In this paper we present the development of the expression component of the emoting system. We focus on its dynamical properties that use a spring model for smooth transitions between emotion expressions over time. This novel method compensates for varying utterance frequency and prediction errors coming from the emotion recognition component. We also describe the input the dynamical expression component receives from the emotion detection component, the development and validation of the output comprising of the gestures instantiated in the robot, and the implementation of the system. We present results from a human validation study that shows people perceive the robot gestures, generated by the system, as expressing the emotions in the speech content. Also, we show that people's perceptions of the accuracy of emotion expression is significantly higher for a mass-spring dynamical system than a system without a mass-spring when specific detection errors are present. We discuss and suggest future developments of the system and further validation experiments. This paper is part of a larger project to develop a prototype for a socially assistive robot for PD persons. The goal is to present the technical implementation of one robot capability: emotion expression.