The Rise of Emotion-aware Conversational Agents: Threats in Digital Emotions

Abstract

A future where the conversation with machines can potentially involve mutual emotions between the parties may be not so far in time. Inspired by the episode of Black Mirror "Be Right Back'' and Replika, a futuristic app that promises to be "your best friend'', in this work we are considering the positive and negative points of including an automated learning conversational agent inside the personal world of feelings and emotions. These systems can impact both single individuals and society, worsening an already critical situation. Our conclusion is that a regulation on the artificial emotional content should be considered before actually going beyond some one-way-only limits.