A Wearable Real-Time Character Recognition System Based on Edge Computing-Enabled Deep Learning for Air-Writing

Abstract

Nowadays, wearable sensors play a vital role in the detection of human motions, innovating an alternate and intuitive form in human-computer interaction (HCI). In this study, we present a novel real-time wearable system for finger air-writing recognition in three-dimensional (3D) space based on the Arduino Nano 33 BLE Sense as an edge device, which can run TensorFlow Lite to realize recognition and classification on the device. This system enables users to have the freedom and flexibility to write characters (10 digits and 26 English lower-case letters) in free space by moving fingers and uses a deep learning algorithm to recognize 36 characters from the motion data captured by inertial measurement units (IMUs) and processed by a microcontroller, which are both embedded in an Arduino Nano 33 BLE Sense. We prepared 63000 air-writing stroke data samples of 35 subjects containing 18 males and 17 females for convolutional neural network (CNN) training and achieved a high recognition accuracy at 97.95%.