An intelligent system to automatically generate video-summaries for accessible learning objects for people with hearing loss

Abstract

The latest estimates of the World Health Organization point that approximately 5% of world’s population presents disabling hearing loss (328 million adults and 32 million children). This situation becomes very complex in developing countries, where children with hearing loss and deafness rarely have access to schooling. On those grounds, in this paper we present an intelligent system to automatically generating video-summaries and captioning in sign language with the aim of creating accessible learning objects for children and youth with disabling hearing loss. Through our intelligent system, we propose a methodology that allows performing agile consumption of educational content (Learning Accessible Objects) that use videos. With the aim of determining the real feasibility of our proposal, we have performed a preliminary experiment with 7 educational videos of history, natural sciences and mathematics for deaf children and youth from 8 to 12 years. A team of 6 experts in sign language has evaluated several aspects our proposal (coherence of the summaries and captions, quality of the synchronization between the explicative elements in sign language and video execution, possible lost keywords/meanings, …).