E- subtitles: Emotional subtitles as a technology to assist the deaf and hearing-impaired when learning from television and film

Abstract

The use of subtitling to represent audio dialogue of film and television presentations is well known and understood. In context of learning technologies, Deaf and hearing-impaired students use subtitles to read dialogue and identify characters. Some simple manipulations such as the increasing of text sizes and the changing of colour are often made available. To date, however, little work has been directed towards the adaptation of subtitles to indicate the emotional nuances of dialogue. Such adaptation could potentially yield significant benefits to hearing-impaired and Deaf students who wish to study and learn from film and television presentations. In this paper, we introduce a model and a system that demonstrates the presentation of subtitles which depict (some of) the emotions behind the words used, as well as the words themselves. The system also provides viewers with the ability to personalise their interaction with subtitles, enabling them to gain the maximum benefit from the subtitle presentation. © 2006 IEEE.