Collection and classification of gestures from people with severe motor dysfunction for developing modular gesture interface

Abstract

This study explores gesture controlled user interfaces for people with severe motor function disabilities stemming from cerebral palsy, quadriplegia, and traumatic brain injury. As a result of their disabilities (involuntary movement and spasticity), it is nearly impossible for these individuals to use conventional interface switches and other input devices to access and use a computer. The ultimate objective of this work is to provide these users with user-friendly, cost-effective gesture controlled interfaces that will enable them to comfortably operate a personal computer. We have now succeeded in developing a non-contact, non-restraining interface based on an off-the-shelf image range sensor that recently became available. In addition, we surveyed a large number of disabled subjects and compiled a fairly exhaustive collection of gestures that these users are capable of making, and classified these voluntary movements in terms of the body part involved. Finally, a series of recognition modules have been developed that are optimized to recognize the gestures associated with each body part (hand, head, leg, etc.). This paper provides an overview of the gesture data collection and classification processes, and discusses the development of the recognition modules.