Road Pothole Recognition and Size Measurement Based on the Fusion of Camera and LiDAR

Abstract

In intelligent driving systems, accurate detection and measurement of the three-dimensional (3D) dimensions of road potholes are essential for optimizing decision-making processes, including deceleration and obstacle avoidance. To address the challenges posed by unreliable depth information from images and the impracticality of large LiDAR point cloud data in real-time applications—issues that can result in false positives or missed detections—we propose a fusion-based approach that integrates images and LiDAR point clouds for measuring pothole dimensions. Initially, we extract ground LiDAR point clouds from raw data through statistical filtering and ground segmentation. Subsequently, we generate a frustum as a two-dimensional (2D) region of interest to identify potential pothole areas within the ground LiDAR point clouds. We then utilize roughness feature description and the Mean-shift clustering algorithm to extract precise sets of pothole points, which allows us to determine the depth, length, width, and relative coordinates of the potholes with respect to the vehicle. Finally, experiments conducted using the open-source KITTI Road dataset and real vehicle data reveal that our method accurately delineates pothole contours, achieving a 27.4% improvement in accuracy over single LiDAR methods and reducing the average processing time by 88.2%. In practical scenarios, the relative error in size measurement is generally within 15%, with an average data processing time of 45.6 ms per frame, thereby satisfying the system’s real-time requirement of 100 ms per frame.