Factors affecting the integration of auditory and visual information in speech: The effect of vowel environment

Abstract

In the McGurk effect, observers typically report the illusory syllable /da/ when they hear the auditory syllable /ba/ presented in synchrony with a video display of a talker saying /ga/. While the effect itself has been well established, there is still little research on the conditions under which the effect occurs. In the experiment reported here, the number of illusory /d/ responses to the auditory /b/-visual /g/ combination is examined in three vowel environments: /a/, /i/, and /u/. The results of this study indicate that the magnitude of the illusion is not the same across different vowel environments. It appears to be strongest for the /i/ vowel, moderate for /a/, and almost nonexistent for /u/. The results thus show that vowel environment is an important factor in determining the magnitude of the McGurk effect, which needs to be considered in accounts of auditory-visual integration during speech perception. [Work supported by NIH.]