Preliminary Investigation of Text Entry Method with Haptic Feedback from Real Object Surfaces Estimated Using Hand Tracking on HMD

Abstract

In virtual reality systems, users enter text by selecting virtual keys with the fingers. A virtual keyboard is displayed in mid-air and thus does not provide haptic feedback. To address this problem, we present a text entry method that uses the surfaces of real objects around the user to provide haptic feedback. A real object surface touched a hand is recognized using the position and posture of the hand acquired by a hand-tracking sensor on a head-mounted display; a virtual keyboard is placed on that surface to provide haptic feedback. We performed a pilot study to compare text entry performance when the virtual keyboard was placed in mid-air, on a wall, on a desk, and on the user's thighs. The result shows that each virtual keyboard placement and the presence/absence of haptic feedback did not affect input performance.