Serial dependence in emotion perception mirrors the autocorrelations in natural emotion statistics

Abstract

A critical function of the human visual system is to track emotion accurately and continuously. However, visual information about emotion fluctuates over time. Ideally, the visual system should track these temporal fluctuations—these “natural emotion statistics” of the world—over time. This would balance the need to detect changes in emotion with the need to maintain the stability of visual scene representations. The visual system could promote this goal through serial dependence, which biases our perception of facial expressions toward those seen in the recent past and thus smooths our perception of the world. Here, we quantified the natural emotion statistics in videos by measuring the autocorrelations in emotional content present in films and movies. The results showed that observers’ perception of emotion was smoothed over ∼12 seconds or more, and this time-course closely followed the temporal fluctuations in visual information about emotion found in natural scenes. Moreover, the temporal and feature tuning of the perceptual smoothing was consistent with known properties of serial dependence. Our findings suggest that serial dependence is introduced in the perception of emotion to match the natural autocorrelations that are observed in the real world, an operation that could improve the efficiency, sensitivity, and stability of emotion perception.