Seeing or hearing? Perceptual independence, modality confusions, and crossmodal congruity effects with focused and divided attention

Abstract

Observers were given brief presentations of pairs of simultaneous stimuli consisting of a visual and a spoken letter. In the visual focused-attention condition, only the visual letter should be reported; in the auditory focused-attention condition, only the spoken letter should be reported; in the divided-attention condition, both letters, as well as their respective modalities, should be reported (forced choice). The proportions of correct reports were nearly the same in the three conditions (no significant divided-attention decrement), and in the divided-attention condition, the probability that the visual letter was correctly reported was independent of whether the auditory letter was correctly reported. However, with a probability much higher than chance, the observers reported hearing the visual stimulus letter or seeing the spoken stimulus letter (modality confusions). The strength of the effect was nearly the same with focused as with divided attention. We also discovered a crossmodal congruity effect: Performance was better when the two letters in a stimulus pair were the same than when they differed in type.