Hand gesture recognition by using 3DCNN and LSTM with adam optimizer

Abstract

A two-step hand gesture recognition system is proposed to classify gestures from different subjects performed under widely varying lighting conditions. First, 3D Convolutional neural network are fine-tuned to classify each hand gesture. Then, the fine-tuned 3D Convolutional neural network are used to learn spatio-temporal features for Long short-term memory automatically. We also perform spatiotemporal data augmentation for more effective training to reduce potential overfitting. In addition, Adam optimizer is employed to improve training speed in both steps. On the VIVA challenge dataset, our method achieves a correct classification rate of 94.5%, and experimental result shows that Adam optimizer outperforms the most commonly used optimizer SGD. Moreover, our system has strong robustness in different lighting conditions.