A Spatiotemporal Deep Learning Architecture for Road Surface Classification Using LiDAR in Autonomous Emergency Braking Systems

Abstract

This paper proposes a spatiotemporal architecture with a deep neural network (DNN) for road surface conditions and types classification using LiDAR for autonomous emergency braking (AEB) systems. It is known that LiDAR provides information on the reflectivity and number of point clouds depending on a road surface. Thus, this paper utilizes the characteristics of LiDAR and the vehicle's speed obtained from in-vehicle sensors for classifying front road surface conditions for the AEB system application. First, we divided the front road area into four subregions to use spatial information from the vehicle's speed. Second, we constructed feature vectors using each subregion's reflectivity, number of point clouds, and in-vehicle information. Third, the DNN classifies road surface conditions and types for each subregion. Finally, the output of the DNN feeds into the spatiotemporal process to make the final classification reflecting vehicle speed and probability given by the outcomes of softmax functions of the DNN output layer. To validate the effectiveness of the proposed method, we performed a comparative study with five other algorithms. With the proposed DNN, we obtained the highest accuracy of 98.0% and 98.6% for two subregions near the vehicle. In addition, we implemented the proposed method on the NVIDIA Jetson TX2 board to confirm that it is applicable in real-time.