Algorithmically Enhanced Wearable Multimodal Emotion Sensor

Abstract

Sensing diverse human emotions offers significant potential for understanding deeper cognitive processes and diagnosing neurological diseases. However, capturing the full spectrum of emotions remains a significant challenge due to their intricate and subjective nature. Traditional emotion-sensing techniques often focus on singular emotions, limiting their ability to grasp the complexity of emotional experiences. Herein, a fully printed, organic wearable sensor capable of multimodal emotion sensing by noninvasively monitoring physiological indicators such as heart rate, breathing patterns, and voice signatures is presented. The recorded signals are processed using a long short-term memory (LSTM) neural network, achieving over 91% classification accuracy in distinguishing different emotions through a data fusion approach, showing >9% enhancement in accuracy as compared to feature fusion. As a proof of concept, Q-learning is implemented with the data to simulate emotional responses in a robotic model. The study's approach provides a pathway to understanding complex human emotions and enhances the capabilities of effective human–machine interaction.