Lip Reading Using Neural Networks and Deep Learning

Abstract

Lip reading is a method for understanding words or speech without the use of audio through visual interpretation of face, mouth, and lip movement. This activity is challenging since different dictions and speech articulations are used by different people. Using deep learning and other techniques, this research validates the usage of machine learning. using neural networks to create a system for automatically reading lips. Two different CNN architectures were trained on a portion of the dataset. On the basis of their ability to accurately predict words, the trained lip reading models were assessed. A online application for real-time word prediction using the top performing model. Keywords: lip reading, computer vision, deep learning, convolutional neural network, web application, object detection.