A multimodal illusion of force improves control perception in above-surface gesture: Elastic zed-zoom

Abstract

Emerging above-surface technology is an opportunity to exploit interaction spaces above a device’s surface; however, the resulting loss of the proprioceptive feedback available from on-surface interactions degrades the user’s sense of control and precision. We asked whether a pseudohaptic illusion (PHI) could help: a sense of force in the absence of actual contact, induced by manipulating the relation of body motion to graphical and auditory cues. To examine the value of above-surface PHIs, we used a zooming microtask, because finger occlusion impedes current implementations on small displays such as smartwatches. In a qualitative study (N = 12), we were able to trigger a physical illusion most often described as elasticity in 92% of participants through physical control/graphical display (C/D) manipulation, and that audio cues significantly strengthened the illusion. Participants experiencing this PHI reported improved sense of control when zooming, and found the interaction’s physicality natural.