Haptic telepresence system for individuals with visual impairments

Abstract

In this paper, we propose a new haptic telepresence system for individuals with visual impairments (VIs) using a red-green-blue-depth (RGB-D) sensor and a haptic device. Recent improvements in RGB-D sensors have enabled real-time access to 3D spatial information. However, the real-time representation of the tangible haptic experience has not been sufficiently challenged. Thus, the proposed system addresses the telepresence of remote 3D information using an RGB-D sensor through video encoding and 3D depth-map enhancement. In the implemented system, the Kinect sensor from Microsoft is an RGB-D sensor that provides depth and 2D color images at a rate of approximately 30 fps. The Kinect depth data frame is buffered, projected into a 3D coordinate system with a resolution of 640 × 480 pixels, and then is transformed into a 3D map. To verify the benefits of the proposed video content adaptation method for individuals with VIs, in this study, we implemented a '2D plus depth map'-based haptic telepresence system; it conducts user experience experiments and presents the results of user response time.