Hand tracking using wearable wrist imaging

Abstract

The human hand is dexterous and versatile, allowing it to interact with physical and virtual environments. The ability to track hand movements during daily activities could be of use in the development of spatial computing, virtual and augmented reality, robotics and prosthetics. However, current techniques based on cameras, strain and inertial sensors, and electromyography sensors have limited view angles and hand positions, have constrained hand activities and sensations, and can track only discrete hand gestures, respectively. Here we report a fully integrated, wireless and wearable ultrasound imaging wristband that is combined with an artificial intelligence algorithm. The wristband can continuously track arbitrary hand configurations of the five fingers and the palm in real time during daily activities with a delay of less than 120 ms. We show that the wristband can be used for intuitive and versatile controls in virtual-reality and robotic-hand applications.