Hedonic Multitasking: The Effects of Instrumental Subtitles During Video Watching

Abstract

Hedonic videos are often accompanied by instrumental text/subtitles. However, the neurophysiological effects of task-switching between instrumental text processing and audiovisual processing during hedonic video watching remains unexplored. We investigated these effects using changes in pupil diameter and theta-band spectral power as indices of cognitive load, and self-reported and automated facial expression-based measures of emotion. We found that the presence of instrumental subtitles subjectively improved comprehension without negatively impacting cognitive and emotional response. These findings contribute to the literature by showing that in this specific context, multitasking does enhance the user experience. Further research is needed to explore the extent to which instrumental text is useful, and how its effects change in accordance with audio presence during passive hedonic experiences across various contexts.