Estimating timing of specific motion in a gesture movement with a wearable sensor

Abstract

Wearable devices with motion sensors, such as accelerometers and gyroscopes, are expected to become popular. There is a lot of research on recognizing gestures using data obtained from motion sensors. A gesture is a one-off motion and its trajectory, i.e., the waveform of the gesture part, is considered to be important. After segmenting the data, gestures are recognized using a template matching method. However, there is no method to accurately detect when a specific action is performed during a gesture. Although it is possible for a player in a game to perform a throwing motion by the user performing a pitching action, it is difficult to reflect a particular moment, such as the user’s release point, in the player. The authors previously proposed a method using a wrist-worn sensor for determining the moment of touching a card in competitive karuta (a Japanese card game) and developed a system that judges the player who took a card first in a competitive karuta match. As reported in this paper, we improved the estimation method to apply our study to a variety of gestures other than those in competitive karuta and propose a method of detecting the timing of a specific action. Our system was evaluated for three types of release points, baseball throws, basketball free throws, and dart throws, with 11 subjects who had an accelerometer and a gyroscope attached to the wrist. The percentage of release point estimation errors of 12 ms or less was determined to be 100% for baseball, 87.6% for basketball, and 91.1% for darts.