A NOVEL EMOTION DETECTION SYSTEM FOR SERVICE ROBOTS USING CONVOLUTIONAL NEURAL NETWORKS

Abstract

The field of service robotics still faces numerous design challenges, with human-robot integration being one of the most significant and complex. This challenge encompasses both physical and emotional aspects, making it imperative to find effective solutions. Our research group has been evaluating various algorithms on our robotic platform, ARMOS TurtleBot. One such algorithm, recently developed by our team, is a scheme for the identification of human emotions from facial characteristics. Although this scheme has achieved a 92% success rate in controlled laboratory conditions, its performance drops significantly in less favorable conditions, such as low light or partially covered faces. To address this issue, we propose a complementary loop to estimate the emotional state of a person from their voice. To achieve this, we trained a convolutional neural network (CNN) with spectral images generated from audio samples characteristic of seven emotions. Our results showed that this model achieved a 69% hit rate, and when combined with our facial recognition algorithm, the overall performance of the system improved to 96.5%. The integration of voice and facial recognition algorithms enhances the reliability and accuracy of emotion detection, making our robotic platform more useful and effective in real-world applications.