Visualization of non-verbal expressions in voice for hearing impaired: Ambient font and onomatopoeic subsystem

Abstract

Generally, a hearing impaired person is supported by staffs to take a note while hearing a lecture. However, the lecture note cannot express a tone of the teacher's voice. Further, non-verbal information such as a chatting voice in a classroom, speed, loudness and tone of speaker's voice are also difficult to express. As a result, it is difficult for a hearing impaired person to feel the atmosphere in the classroom. In this study, we develop a system to inform atmosphere in the classroom to a hearing impaired person. The system utilizes expression techniques used in Japanese cartoons; they are "Ambient Font", "Balloon & Symbols" and "Onomatopoeic Word". These techniques enable us to inform to the hearing impaired person not only the textual information but also the non-verbal information. © 2012 Springer-Verlag.