Affective communication enhancement system for locked-in syndrome patients

Abstract

Patients with Locked-In Syndrome such as people with Amyotrophic Lateral Sclerosis (ALS) rely on technology for basic communication. However, available Augmentative and Alternative Communication (AAC) tools such as gaze-controlled keyboards have limited abilities. In particular, they do not allow for expression of emotions in addition to words. In this paper we propose a novel gaze-based speaking tool that enable locked-in syndrome patients to express emotions as well as sentences. It also features patient-controlled emotionally modulated speech synthesis. Additionally, an emotional 3D avatar can be controlled by the patient to represent emotional facial-expressions. The systems were tested with 36 people without disabilities separated into an affective group - full control of emotional voice, avatar facial expressions and laugh - and a control group - no emotional tools. The study proved the system’s capacity to enhance communication for both the patient and the interlocutor. The emotions embedded in the synthesized voices were found recognizable at 80% on the first trial and 90% on the second trial. The conversation was perceived as more natural when using the affective tool. The subjects felt it was easier to express and identify emotions using this system. The emotional voice and the emotional avatar were found to help the conversation. This highlights the needs for more affective-driven communicative solutions for locked-in patients.