Touchless typing with head movements captured in thermal spectrum

Abstract

Many physically challenged people are unable to operate standard electronic equipment or computer input devices. They need special assistive technologies and one of the options is the head operated interface. Face-oriented algorithms often assume a particular level of lighting with adequate intensity and spatial configuration. In the paper, we propose a thermal-imaging-based algorithm of head operating typing. It does not assume the visible light illumination. We investigated, in context of thermal imagery, several contemporary general purpose object detectors known to be accurate in case of images captured by the visible light camera. Then, a selected face detector is employed in the head operated interface analysing head movements in the thermal spectrum. The attention has been focused on the problem of touchless typing which is performed in the existing solutions either through the camera mouse or through traverse procedure with an addition mechanism (like eye blink or mouth open) needed for clicking events in both cases. Our novel solution for touchless typing with head movements combines the thermal imaging for capturing user action with the hierarchical letter selection procedure. The solution employed allows to reach any alphabet character in just three steps, i.e. with directional head movements, without the need of any additional mechanisms for clicking events.