Hearing How Smooth It Looks: Selective Attention and Crossmodal Perception in the Arts

Abstract

A broad range of behavior is associated with crossmodal perception in the arts. Philosophical explanations of crossmodal perception often make reference to neuroscientific discussions of multisensory integration in selective attention. This research demonstrates that superior colliculus plays a regulative role in attention, integrating unique modality specific visual, auditory, and somatosensory spatial maps into a common spatial framework for action, and that motor skill, emotional salience, and semantic salience contribute to the integration of auditory, visual, and somatosensory information in ordinary perceptual contexts. I present a model for multisensory integration in our engagement with artworks derived from a diagnostic recognition framework for object recognition and a biased competition model for selective attention. The proposed model attributes a role to superior colliculus in a broader fronto-parietal attentional network that integrates sensory information, primes perceptual systems to the expectation of stimulus features salient to particular sensorimotor or cognitive tasks at particular locations, and inhibits the perception of task irrelevant distracters. I argue that this model demonstrates that crossmodal effects are the rule not the exception in perception and discuss ways in which it explains a range of crossmodal effects in our engagement with pictures, dance, and musical performances. Can music sound like a gesture or a bodily posture? Can sense modalities be crossed that way? There certainly is plenty of room for doubt about it? 1 A broad range of behavior is associated with crossmodal perception in the arts: we hear the sounds of a movie emanating from the location of the visual event it depicts, not one determined by the arrangement of the speakers that surround us in the theater; the perceived duration of a note we hear is influenced by the gesture with which we see it performed; motor skill and perceived task difficulty can influence the way we hear music and see the