An intelligent multimedia E-learning system for pronunciations

Abstract

The proposed system relates to an interactive scoring system for learning a language, in which a means such as a web camera is used to capture the learners lip movements and then a score is given by making a comparison with images stored in the database. The images stored in the database are those previously recorded by a teacher. By means of the scoring system, the learner can identify and rectify pronunciation problems concerning the lips and tongue. The system also records sounds as well as images from the student. The proposed system processes this data with multimedia processing techniques. With regard to the interactive perspective, a user-friendly visual interface was constructed to help learners use the system. The learners can choose the words they want to practice by capturing their lip image sequences and speech. The lip region image sequences are extracted automatically as visual feature parameters. Combining the visual and voice parameters, the proposed system calculates the similarity between a learners and a teachers pronunciation. An evaluation score is suggested by the proposed system through the previous similarity computation. By this learning process, learners can see the corresponding lip movement of both themselves and a teacher, and correct their pronunciation accordingly. The learners can use the proposed system to practice their pronunciation as many times as they like, without troubling the human teacher, and thus they are able to take more control of improving their pronunciation. © Springer-Verlag Berlin Heidelberg 2007.