Rules of intersensory integration in spatial scene analysis and speechreading

Abstract

Audiovisual interactions observed in the ventriloquism and McGurk illusions were compared, using the same materials. The ventriloquism effect was estimated through a discordance detection task and the McGurk illusion through an identification task. The stimuli were visually displayed on a screen located in front of the participants' head and acoustically delivered through one of nine hidden loudspeakers located from straight ahead (0°) to 80° (20° steps) left and right. The speaker's face was either upright or inverted. The ventriloquism effect was affected by the degree of spatial separation, but unaffected by upright vs. inverted presentation of the face, or by the congruency of the stimuli. The McGurk illusion was of the same size whatever the loudspeaker location but was reduced by face inversion. The differences in the spatial and cognitive rules that govern both interactions are discussed in terms of specific functionality of the underlying mechanisms.