Dissociating stimulus-response compatibility and modality compatibility in task switching

Abstract

Modality compatibility (MC) describes the similarity between the modality of the stimulus and the modality of the anticipated response effect (e.g., auditory effects when speaking). Switching between two incompatible modality mappings (visual-vocal and auditory-manual) typically leads to larger costs than switching between two compatible modality mappings (visual-manual and auditory-vocal). However, it is unclear whether the influence of MC arises before or after task selection or response selection, or affects both processes. We investigated this issue by introducing a factor known to influence response selection, stimulus-response (S-R) compatibility, examining possible interactions with MC. In Experiment 1, stimulus location was task-irrelevant; participants responded manually or vocally to the meaning of visual and auditory colour words presented left or right (Simon task). In Experiment 2, stimulus location was task-relevant; participants responded manually or vocally, indicating the location (left or right) of visual or auditory stimuli, using a spatially compatible versus incompatible mapping rule (“element-level” S-R compatibility). Results revealed independent effects of S-R and modality compatibility in both experiments (n = 40 per experiment). Bayes factors suggested moderate but consistent evidence for the absence of an interaction. Independent effects suggest MC effects arise either before or after response selection, or possibly both. We propose that motor response initiation is associated with anticipatory activation of modality-specific sensory effects (e.g., auditory effects when speaking), which in turn facilitates the correct response in case of modality-compatible mappings (e.g., auditory-vocal) or reactivates, at the task-selection level, the incorrect task in case of modality-incompatible mappings (e.g., visual-vocal).