Emergency Sound Recognition and Direction Indication Using Machine Learning for Individuals with Hearing Loss

Abstract

Sound is a vital means of expression and communication. However, individuals with hearing loss face considerable challenges, particularly in emergency situations. During such events, alerts and critical information are typically conveyed through sirens or verbal commands, which are ineffective for those with hearing loss. To overcome this limitation, an attachable or wearable assistive device is necessary. The proposed wearable system in this work consists of (i) microphone arrays embedded in a headband, and (ii) a wristband with a motor vibrator and display attached to it. The neural network algorithm, specifically the multi-layer perceptron model, is used for selective emergency sound (dog bark, car honk and siren) recognition and classification. Once an emergency sound is detected by the microphone arrays and classified by the algorithm, a vibration signal is transmitted to the motor, while the direction of the sound is simultaneously indicated on the wristband display. The trained model and real-time implementation achieved an accuracy of 90% and 77%, respectively.