Audiovisual temporal integration for complex speech, object-action, animal call, and musical stimuli

Abstract

Understanding the mechanisms underlying the human perception of synchrony for simple and complex audiovisual stimuli represents an important, but as yet unresolved, issue in the field of cognitive science. Many questions regarding the processes involved in the temporal integration of auditory and visual stimuli that give rise to a synchronous audiovisual experience of everyday events are still open for research. This chapter outlines what is currently known about the mechanisms of audiovisual temporal perception and reviews the results of a series of studies of temporal perception using complex audiovisual stimuli. To date, two characteristics of the audiovisual temporal window of integration have been shown to be relatively consistent across the majority of studies: (1) It has a width on the order of several hundred milliseconds and (2) it is asymmetrical, being larger when the visual-stimulus leads than when it lags. We provide an overview of research demonstrating that the temporal window of audiovisual integration for complex stimuli is modulated by the type, complexity, and properties of the particular experimental stimuli used, the familiarity of the observer with the stimuli presented, the degree of unity of the auditory- and visual-stimulus streams (for the case of speech stimuli), and the orientation of the visual stimulus (again for the case of speech stimuli).