Recent Multimodal Communication Methodologies in Phonology, Vision, and Touch

Abstract

Due to the innovation and technology’s capacity in recent years, multimodal communication can help humans to communicate and interact in more basis senses simultaneously using technological innovations. However, there are still many difficulties and challenges for achieving the multimodal communication systems in phonology, vision, and touch. In this paper, we present recent multimodal communication methodologies in different aspects. To begin with, we review the multimodal communication systems in phonology, phonetics and light. Then, we investigate the multimodal communication works in vision using spatial augmented reality (SAR), computer vision, and interactive digital multimedia. Next, we introduce the multimodal communication researches in touch using haptic sensors and related-technological innovations. After that, we propose the qualitative research of the extended augmented reality and interactive experience-based system using use in-depth interviews. Our experimental results have revealed that the presented system works well and increases users’ satisfaction levels. After examining the approaches for integrated multimodal communication in phonology, vision, and touch presented in recent years, we can understand the interactions of human with recent innovations.