Direct finger manipulation of 3D object image with ultrasound haptic feedback

Abstract

In this study, we prototype and examine a system that allows a user to manipulate a 3D virtual object with multiple fingers without wearing any device. An autostereoscopic display produces a 3D image and a depth sensor measures the movement of the fingers. When a user touches a virtual object, haptic feedback is provided by ultrasound phased arrays. By estimating the cross section of the finger in contact with the virtual object and by creating a force pattern around it, it is possible for the user to recognize the position of the surface relative to the finger. To evaluate our system, we conducted two experiments to show that the proposed feedback method is effective in recognizing the object surface and thereby enables the user to grasp the object quickly without seeing it.