A Reinforcement Learning Framework to Foster Affective Empathy in Social Robots

Abstract

This work aims to endow a social robot with the ability to mimic affective empathy, which represents the skill of inferring the other’s emotional state and mirroring the detected emotion. In this direction, we first designed a set of 52 facial expressions that may be representative of the primary emotions. Leveraging on the idea that the robot should learn the matching between emotion and its expressions directly from the final users, we modeled a deep reinforcement learning algorithm, and we recruited 105 users to train it, by rating the coherence of the robot’s facial expressions on a web interface. A total of 22251 facial configurations were generated by the algorithm and rewarded by the pool of participants. The results proved that the algorithm exploited every facial configuration, converging towards a subset of 6 facial expressions at the end of the teaching process. Thus, we tested the trained empathetic model on a real robot (i.e. CloudIA robot) in a conversation scenario. The results collected through the interview and the questionnaires’ analysis highlighted a general tendency of preferring the empathetic behavior of the robot, more than the not empathetic one. Namely, the robot endowed with empathetic behavior was perceived as more human-like and aware of the context of the interaction.