Keep in Touch: Combining Touch Interaction with Thumb-to-Finger μGestures for People with Visual Impairment

Abstract

We present a set of 8 thumb-to-finger microgestures (TTF μGestures) that can be used as an additional modality to enrich touch interaction in eyes-free situations. TTF μGestures possess characteristics especially suited for people with visual impairment (PVI). They have never been studied specifically for PVI to improve accessibility of touchscreen devices. We studied a set of 33 common TTF μGestures to determine which are feasible and usable without seeing while the index is touching a surface. We found that the constrained position of the hand and the absence of vision prevent participants from being able to efficiently target a specific phalanx. Thus, we propose a set of 8 TTF μGestures (6 taps, 2 swipes) balancing resiliency (i.e., low error-rate) and expressivity (i.e., number of possible inputs): as a dimension combined with the touch modality, it would realistically multiply the touch command space by eight. Within our set of 8 TTF μGestures, we chose a subset of 4 μGestures (2 taps and 2 swipes) and implemented an exploration scenario of an audio-tactile map with a raised-line overlay on a touchscreen and tested it with 7 PVI. Their feedback was positive on the potential benefits of TTF μGestures in enhancing the touch modality and supporting PVI interaction with touchscreen devices.