Grounded approach for understanding changes in human emotional states in real time using psychophysiological sensory apparatuses

Abstract

This paper discusses the technical and philosophical challenges that researchers and practitioners face when attempting to classify human emotion based upon raw physiological data. It proposes the use of a representational learning approach that adopts techniques from industrial internet of things (IoT) solutions. It applies this approach to the classification of emotional states using functional near infrared spectroscopy (fNIRS) sensor data. The algorithm used first pre-processes the data using a combination of signal processing and vector quantization techniques. Next, it found the optimal number of natural clusters within human emotional states and used these as the target variables for either shallow or for deep learning classification. The deep learning variant used a Restricted Boltzmann Machine (RBM) to form a compressive representation of the input data prior to classification. A final single layer perception model learned the relationship between the input and output states. This approach would be useful for detecting real-time changes in human emotional state. It is able automatically create emotional states that are both highly separable and balanced. It is able to distinguish between low v. high emotional states across all tasks (F1-score of 71.4%) and is better at forming this distinction for tasks intended to elicit higher cognitive load such as the Tetris video game (F1-score of 87.1%) or the Multi Attribute Task Battery (F1-score of 77%).