ivision: An assistive system for the blind based on augmented reality and machine learning

Abstract

In this paper, an assistive system iVision for the blind is proposed, which can solve one of the main problems faced by the visually impaired. We use augmented reality technology to create a three-dimensional model of the surrounding space and machine learning model to find the object of interest in the image taken by the camera. Then we combine these data to get the actual spatial location of the objects. This location information can be used by visually impaired people to search for objects of interest and avoid dangerous obstacle. The system uses speech and audio to feedback the spatial location to the user. Voice user interfaces (VUIs) is designed for iVision which allow the user to interact with a system through voice or speech commands. We verified the usability of the proposed system in evaluation experiments and noted its innovations and limitations.