A system for assisting the visually impaired in localization and grasp of desired objects

Abstract

A prototype wearable visual aid for helping visually impaired people find desired objects in their environment is described. The system is comprised of a head-worn camera to capture the scene, an Android phone interface to specify a desired object, and an attention-biasingenhanced object recognition algorithm to identify three most likely object candidate regions, select the best-matching one, and pass its location to an object tracking algorithm. The object is tracked as the user’s head moves, and auditory feedback is provided to help the user maintain the object in the field of view, enabling easy reach and grasp. The implementation and integration of the system leading to testing of the working prototype with visually-impaired subjects at the Braille Institute in Los Angeles (demonstration in the accompanying video) is described. Results indicate that this system has clear potential to help visually-impaired users in achieving near-real-time object localization and grasp.