ERUSLR: A new Turkish sign language dataset and its recognition using hyperparameter optimization aided convolutional neural network

Abstract

Sign language is one of the most important tools of communication for deaf-and-dumb individuals who have lost their linguistic and auditory abilities. Significant problems may arise in situations where the sign language required to clearly understand deaf-and-dumb individuals is not known. More importantly, the failure to understand the disabled individuals who try to access emergency health services at a health institution may have fatal consequences. In this study, firstly, a new dataset was created with the frequently used words in the emergency department of hospitals. 25 words were repeated multiple times by 49 handicapped individuals where the videos were recorded from different angles. This dataset, named Erciyes University Sign Language Recognition (ERUSLR), contains 13186 samples. Classification experiments were performed by using the ERUSLR dataset. Sign language recognition can be realized by a convolutional neural network (CNN), which is frequently used for classification problems. Rather than developing a new CNN model, transfer learning, a more effective method, is preferred. Therefore, a GoogLeNet-based CNN model was created by transfer learning from the GoogLeNet pre-trained model. Another factor that increases the performance of a CNN model is the optimization of its training parameters. Global and heuristic search methods are typically used in parameter optimization to save time. In this study, both grid search (GS), random search (RS), and genetic algorithm (GA) methods were used to optimize the training parameters of the GoogLeNet-based CNN model. According to the experimental results, the GA supported GoogLeNet-based CNN model is more successful (with a success rate of 93.93%) than the other methods.