Comic readers’ emotion estimation using bio-signals by supervised and unsupervised learnings

Abstract

In this paper, we investigate estimating readers' emotions during reading comics from bio-signals of brainwave, heartbeat, pupil diameter and gazing point. We collect the bio-signals from 11 subjects while they are reading comics, and ask them to answer a questionnaire after reading comics on raised emotions in seven categories including "pleased", "excited", "astonished", "afraid", "frustrated", "sad" and "calm". For the analysis, we use DNN (Deep Neural Network) as a supervised learning method as well as AE (Autoencoder) as an unsupervised one. The questionnaire responses are taken as the correct labels of the raised emotions for DNN, and used to evaluate the clustering results of them by AE. As the results of the analysis, we obtain a high F-score for each emotion estimation by DNN, and find several clusters dominantly including a particular emotion by AE. It suggests that comic readers' emotions can be estimated from the bio-signals by both supervised and unsupervised learnings.