Seeing glee but hearing fear? emotional McGurk effect in Swedish

Abstract

Video and audio recordings of emotional expressions of a male speaker were used to construct conflicting stimuli in order to perform a McGurk experiment. The results are compared to an almost identical experiment with a female speaker, described in Abelin (2007) in order to see if the earlier results were reproduced. The results from both experiments show that in the McGurk condition the visual channel was in general more reliable than audio at conveying emotions, i. e. the results from Abelin (2007) were reproduced. Most frequently the listeners interpreted in accordance with the visual stimuli, or they heard an emotion which was not present in either video or audio. The emotions happiness and anger showed a dominance for visual perception, and the emotions fear, and to some extent surprise showed a preference for auditive perception, in the McGurk condition. No emotional dimension, i.e. the evaluative (positive-negative) was connected specifically to the visual or the auditive modality.