Multimodal and multisensory displays for perceptual tasks

Abstract

Introduction The human sensory system allows individuals to perceive the natural environment and maintain situation awareness during complex, dynamic tasks. The human brain can use multiple sensory channels to perceive the contextual data needed to disambiguate incomplete or illusory data provided by any single sense. However, disease, injury, and human designed environments (e.g., workspaces) can restrict sensory data flow from normally available sensory channels and degrade understanding of information perception and situation awareness. Without the multisensory representations available in the natural world, humans in sociotechnical work systems often must control technology while relying on system interfaces that impair maintenance of situation awareness. As technological capabilities to sense data in artificial environments have advanced (e.g., temperature, pressure, speed, altitude, radar, chemical composition, bandwidth), information interfaces have not kept pace with regard to matching the diversity of the sources of signals to the capabilities of the human brain. By exploiting the multiple channels of the human sensory system, multisensory multimodal display designs have the potential to facilitate human perception and overall sociotechnical team performance. This chapter reviews common causes of sensory channel loss and current research on noninvasive sensory displays. The particular focus is on tactile interfaces that can improve performance on perceptual tasks when used in concert with other sensory displays. We also discuss how and why multimodal and multisensory display approaches might be employed.