A fast and flexible computer vision system for implanted visual prostheses

Abstract

Implanted visual prostheses generate visual percepts by electrically stimulating the human visual pathway using an array of electrodes. The resulting bionic vision consists of a spatial-temporal pattern of bright dots called phosphenes. This patient-specific phosphene pattern has low resolution, limited dynamic range and is spatially irregular. This paper presents a computer vision system designed to deal with these limitations, especially spatial irregularity. The system uses a new mapping called the Camera Map to decouple the flexible spatial layout of image processing from the inflexible layout of phosphenes experienced by a patient. Detailed simulations of a cortical prosthesis currently in preclinical testing were performed to create phosphene patterns for testing. The system was tested on a wearable prototype of the cortical prosthesis. Despite having limited computational resources, the system operated in real time, taking only a few milliseconds to perform image processing and visualisations of simulated prosthetic vision.