Non-intrusive measurement of player engagement and emotions - real-time deep neural network analysis of facial expressions during game play

Abstract

Prior research suggests and reveals that there is a correlation between human emotional responses and the subjective qualities of digital interactive experiences. Using facial analysis done by deep neural networks presents a true non-intrusive way of measuring emotional responses and engagement assessed as the desire to continue playing. This paper proposes a tool to measure emotional responses across eight different emotions and in real time of any game. The emotional recognition system achieves an accuracy of 98% and the continuation desire system achieves 93.3% accuracy in a pilot test with a two player game and 78.5% accuracy in a single player game. This forms a strong tool that shows a correlation between emotions and the continuation desire of a player, which can be used to evaluate engagement in games and digital interactive experiences, e.g. in critical stages of development of said content.