A Wearable Indoor Navigation System with Context Based Decision Making For Visually Impaired

Abstract

This paper presents a wearable indoor navigation system that helps visually impaired user to perform indoor navigation. The system takes advantage of the Simultaneous Localization and Mapping (SLAM) and semantic path planning to accomplish localization and navigation tasks while collaborating with the visually impaired user. It integrates multiple sensors and feedback devices as an RGB-D camera, an IMU and a web camera; and it applies the RGB-D based visual odometry algorithm to estimate the user's location and orientation, and uses the IMU to refine the orientation error. Major landmarks such as room numbers and corridor corners are detected by the web camera and RGB-D camera, and matched to the digitalized floor map so as to localize the user. The path and motion guidance are generated afterwards to guide the user to a desired destination. To improve the fitting between the rigid commands and optimal machine decisions for human beings, we propose a context based decision making mechanism on path planning that resolves users' confusions caused by incorrect observations. The software modules are implemented in Robotics Operating System (ROS) and the navigation system are tested with blindfolded sight persons. The field experiments confirm the feasibility of the system prototype and the proposed mechanism. The ability of visually impaired people to access, understand, and explore unfamiliar environment will improve their inclusion and integration into the society. It will also enhance employment opportunities, foster independent living and produce economic and social self-sufficiency [7]. After consulting with the blind individuals in organizations such as the Lighthouse International NY, Computer Center for Visually Impaired People (CCVIP) at CUNY Baruch College, New York Institute for Special Education (NYISE), and the New York State Commission for the Blind and Visually Handicapped (CBVH), we realize that visually impaired people demand an assistive technology that can provide them with safe and smooth way-finding capabilities. Unfortunately, existing related assistive technologies have various drawbacks and limitations. SLAM is a process of building a map of unknown environment while at the same time localizing the robot within the map. As an extension of our previous works [17-20, 37] to further improve the existing techniques in visually impaired user navigation, we present the SLAM based navigation system using multiple sensors which specifically fits the demand of visually impaired user navigation in terms of reliability. Figure 1 shows the system at work. It takes advantage of the SLAM technique to fuse the inputs from multiple sensors and localize the user on the floor plan, and represents the information and guidance in a high level semantic map which contains necessary abstracted information for human being. Path planning for visually impaired is different with those for robots. There are numerical optimal/suboptimal solutions that handle uncertainty and incomplete information for robots. However, as a human-centered system, the navigation system shall seek to provide smooth and reliable user experience while considering the facts that visually impaired users are prone to be confused if conflicted or rigid commands are presented. Thus, we propose a context based decision making mechanism in path planning which fits the need of the navigation system. It takes advantage of the fact that the goal is to reach the destination