Phoneme and Viseme based Approach for Lip Synchronization

Abstract

Phoneme to Viseme mapping has great application in Visual Speech Recognition, Lip Synchronization, Talking Head Applications, movies, news reading, and film industries. Lot of work has been done in area of various face component detection and recognition. Apart from eye detection, ear detection, iris detection etc, lip tracking and lip detection is one of the favourite topics for researchers. Various algorithms and techniques have been implemented so far to achieve better and better performance. Normalized RGB colour scheme, HSV colour model, Lip detection using HUE segmentation and many more techniques have been implemented and are in the boom. All methods are having their own pros and cons. We are aiming to extract out phonemes from speech as well as we extract visual feature i.e. visemes from face by using hue and saturation values. The reason behind the selection of this algorithm is that, it performs well under various illumination conditions, which is the one of the dimension of difficulty in the area of lip detection. We are aiming to carry out the work on in-house database with varying lighting and noisy conditions.