INTEGRATED AUDIO–VISUAL EMERGENCY VEHICLE DETECTION FOR AUTONOMOUS VEHICLES WITH REAL-TIME RESPONSE

Abstract

In modern cities, self-driving cars must notice and respond to emergency vehicles such as ambulances and fire trucks quickly so that they don’t cause accidents or get in the way. Our aim is to develop a system that not only listens or looks, but does both at the same time, making it much more intelligent and accurate than using just one of these approaches. On the audio side, we use Google’s YAMNet to detect the siren sound and then pass this through machine learning models to determine if it is from an emergency vehicle. On the video side, we use the YOLOv8 model to detect several characteristics that make emergency vehicles distinctive in video feeds, such as red or blue flashing lights. To train the system, we created a dataset including 4,000 audio clips and many thousands of images, covering a wide range of traffic and weather conditions. When tested, the system performed much better than those that used sound or visuals alone, particularly in challenging situations such as heavy traffic and poor visibility. Best of all, it works in real time, which means it’s ready for real-world use, helping self-driving cars make way for emergency vehicles safely and legally.