Multi-sensor fusion and deep learning for road scene understanding: a comprehensive survey

Abstract

Autonomous vehicles rely on accurate and robust road scene perception to ensure safe navigation in transient, degraded, and dynamic conditions. This survey synthesizes recent advances in multimodal sensor fusion, deep learning architectures, and intelligent perception strategies that transform autonomous vehicle systems. It examines the complementary roles of LiDAR, radar, RGB/IR cameras, thermal imaging, and hyperspectral sensors, alongside fusion approaches at data, feature, and decision levels that improve reliability in complex environments. State-of-the-art deep learning architectures—including YOLO, DeepLabv3 +, U-Net, GAN-based anomaly detectors and transformer models—are analyzed for core scene understanding tasks: construction zone detection, pavement condition assessment, and hazard identification. Critical challenges include limited annotated datasets, environmental variability, and cross-domain generalization, with emerging solutions that encompass self-supervised learning, cross-modal transformers, and federated training. By linking advances in sensing technologies, AI models, and system-level deployment, the survey provides a structured taxonomy and roadmap for developing intelligent, scalable, and resilient transportation systems, offering researchers and practitioners practical insight for designing robust perception frameworks that strengthen the safety and reliability of intelligent transportation systems (ITS).