A multimodal tablet-based application for the visually impaired for detecting and recognizing objects in a home environment

Abstract

Object recognition solutions for the visually impaired based on a single modality cannot provide optimal performance under all circumstances, since each modality is best suited for particular usage scenarios. An object recognition application for the visually impaired, meant for a RFID-enabled tablet, which combines three approaches - RFID-based, visual-tag computer vision based and non-visual tag computer vision based - into a single piece of software is, therefore, presented in this paper. This solution has the benefits of being portable, accessible, low cost (the user needs only an RFID-enabled tablet, some inexpensive passive RFID tags and some visual tags (which can be printed out for free)) and more robust in a wider range of conditions than the approaches it is comprised of. The application will be adapted to other mobile platforms and devices (e.g., RFID-enabled smartphones) in the future. © 2014 Springer International Publishing.