Classification of Sign Language in Real Time Using Convolutional Neural Network

Abstract

Communication between people is essential for daily life activities. However, humans are created with their own strengths and weaknesses. One of them is the difficulty of communication and interaction for people with hearing and speech impairments. Sign language is a language for people who have difficulty hearing and speaking. However, sign language is not popular in society, and people who have it will have more difficulties. This research aims to classify hand gestures of sign language into letters using a convolutional neural network (CNN). The dataset is obtained from Kaggle, with a total of 34,627 data divided by the ratio of training and testing data of 80:20. From the test results, the letters of the alphabet that can be translated are: A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, S, T, U, V, W, X, Y, and Z. Furthermore, validation accuracy is obtained. In this study, a very high validation accuracy was obtained. The easiest letters to guess are V and N, while the most difficult letters to guess are n, c, j, and z. With different preprocessing, the loss value can be reduced, giving a higher accuracy of 95.4%.