Shapeshifter: Gesture Typing in Virtual Reality with a Force-based Digital Thimble

Abstract

Existing text entry techniques for virtual reality are either slow and error-prone, stationary, break immersion, or physically demanding. We present Shapeshifter, a technique that enables text entry in virtual reality by performing gestures and fluctuating contact force on any opaque diffusely reflective surface, including the human body. For this, we developed a digital thimble that users wear in their index finger. The thimble uses an optical sensor to track the finger and a pressure sensor to detect touch and contact force. In a week-long in-the-wild pilot study, Shapeshifter yielded on average 11 wpm on flat surfaces (e.g., a desk) and 9 wpm on the lap when sitting down, and 8 wpm on the palm and back of the hand when standing up in text composition tasks. A simulation study predicted a 27.3 wpm error-free text entry rate for novice users in transcription typing tasks on a desk.