Extended Mouth/Tongue Gesture Recognition Module for People with Severe Motor Dysfunction

Abstract

We researched gesture interfaces for people with motor dysfunction who cannot use normal interface switches. For this purpose, we have developed nine gesture recognition modules. One of them is the tongue recognition module that detects whether the tongue is in or out for the switch interface. In the process of collecting data on various types of mouth and tongue gestures and user’s demands, we discovered three useful types of mouth-related gestures: mouth open/close, tongue in/out, and movement around the lips. These gestures are also useful when performed simultaneously. We conducted several experiments using data of people with motor dysfunction and inspected the conditions for real use. Furthermore, we conducted a basic experiment with healthy people to recognize three gestures simultaneously.