EyeHear: Smart Glasses for the Hearing Impaired

Abstract

The hearing-impaired experience the sound localization problem, defined as the insufficient ability to estimate the direction of sounds. This problem reduces spatial awareness and amplifies social difficulties such as communicating in adverse listening conditions. Current solutions, such as hearing aids and cochlear implants, fail to solve the problem for a multitude of reasons, but recent work in head-computer interfaces can be applied to deliver a robust solution. Thus, this paper develops a smart glass, named EyeHear, that shows the direction of a speech source and its transcription, all in real-time thanks to a modified speech recognition algorithm and speech recognition system. Algorithm testing proved that the device was accurate enough to be feasible in real-world talking situations. User testing affirms that the target audience favors head-mounted displays and indicates that solving the sound localization problem is beneficial. This device paves the way for future work to extend the device to non-speech sounds. EyeHear has the potential to become an assistive technology for the hearing impaired.