Finger gesture recognition with smart skin technology and deep learning

Abstract

Finger gesture recognition (FGR) was extensively studied in recent years for a wide range of human-machine interface applications. Surface electromyography (sEMG), in particular, is an attractive, enabling technique in the realm of FGR, and both low and high-density sEMG were previously studied. Despite the clear potential, cumbersome electrode wiring and electronic instrumentation render contemporary sEMG-based finger gestures recognition to be performed under unnatural conditions. Recent developments in smart skin technology provide an opportunity to collect sEMG data in more natural conditions. Here we report on a novel approach based on soft 16 electrode array, a miniature and wireless data acquisition unit and neural network analysis, in order to achieve gesture recognition under natural conditions. FGR accuracy values, as high as 93.1%, were achieved for 8 gestures when the training and test data were from the same session. For the first time, high accuracy values are also reported for training and test data from different sessions for three different hand positions. These results demonstrate an important step towards sEMG based gesture recognition in non-laboratory settings, such as in gaming or Metaverse.