Traffic Light Recognition Assistant for Color Vision Deficiency Using YOLO with Multilingual Audio Feedback

Abstract

Drivers with color vision deficiency (CVD) often face difficulty recognizing traffic light colors at intersections. Relying solely on their limited color vision can increase safety risks while driving in urban environments. In the era of technological development, Intelligent Transportation Systems (ITSs) increasingly aim to provide support for traffic users, including individuals with CVD. To address user needs from diverse backgrounds, this study aims to develop a traffic light recognition system that provides offline multilingual audio feedback in Indonesian, Mandarin, and English. The proposed approach introduces a spatial-position inference framework by applying a full-frame traffic light annotation strategy to a YOLOv12 model, enabling traffic light state recognition based on the relative position of active lights rather than relying primarily on color information. This work contributes to reducing reliance on color-based perception traffic signal recognition frameworks tailored for assistive ITS applications targeting users with color vision deficiency. System performance is evaluated to verify its feasibility using a comprehensive dataset consisting of various traffic light conditions, including daytime and nighttime scenarios, varying weather, and different traffic densities. Experimental results show an average detection confidence of approximately 0.73, with a maximum confidence of 0.95 and low processing latency of 0.214 s on a CPU-only configuration. The system has the potential to enhance driving safety for individuals with color vision deficiency by offering an additional intelligent assistive tool instead of replacing standard driving regulations.