Towards Affective BCI/BMI Paradigms – Analysis of fEEG and fNIRS Brain Responses to Emotional Speech and Facial Videos

Abstract

A previous study in the visual domain showed that similar spatial patterns of electroencephalography (EEG) power can be seen when subjects are visually presented with short videos of faces expressing emotions. As a corollary study we examine both the auditory and visual expressions of the same emotions to understand how listeners process auditory-only and visual-only expressions of affec-tive stimuli in functional EEG (fEEG) and functional near-infrared-spectroscopy (fNIRS) signals. For this purpose we utilize fEEG directed transfer functions (DTF) analysis in very short windows of 250 ms which potentially is of interest of brain-computer/machine-interfacring (BCI/BMI) application with utilization of affective (limbic) responses from human brains. The preliminary results with affec-tive paradigms confirm the hypothesis of cortical emotional responses in fEEG and fNIRS.