The effect of multimodal input on L2 learners' reading comprehension: A preregistered eye-tracking study

Abstract

Multimodal materials (e.g., written text supplemented by images and/or audio) are commonplace in language classrooms. While they have been consistently shown to be beneficial for vocabulary acquisition, the efficacy of multimodal input in scaffolding text comprehension is less clear. Conflicting findings have also been reported in terms of the relationship between comprehension and attention to pictures as measured by eye tracking. In this preregistered study, we provide further empirical evidence to this research base by testing a new population, adult beginners, and by including a reading-only, no-image condition as the baseline. In a counterbalanced within-subject design, 65 learners of Spanish were exposed to different parts of a story on an eye tracker under three experimental conditions: reading only (RO), reading + image (RI), and reading + image + audio (RIA). Results revealed that comprehension was higher in the RIA and RI than in the RO condition, indicative of the usefulness of the audio and/or the image. The number of looks at the images was higher in the RIA condition, confirming that the audio allowed readers to attend to the pictorial information provided. Attention to the image, however, positively predicted comprehension scores only for the RIA condition.