A Comprehensive Vision-Based Model for Commercial Truck Driver Fatigue Detection

Abstract

Fatigue driving is a primary reason for traffic accidents for commercial truck drivers. Using new technology to detect fatigue in advance is very important to improve road safety. Most of the existing research on fatigue detection is based on the facial key-points of drivers, which are prone to inaccurate detection and thus affect the accuracy of fatigue judgment in complex driving environments. This paper first proposes an end-to-end fatigue detection model based on the CNN-LSTM network structure. The model does not detect facial key-points directly but extracts the driver’s fatigue-related high-level features in full video image frames through a CNN network. Then, the LSTM network is used to calculate the fatigue probability of the driver on the time-series fatigue features of the multi-frame images. To alleviate the impact of complex driving environments, an attention mechanism for CNN is introduced to help the network to focus on the region of interest. To further eliminate background disturbance, this paper proposes a comprehensive fatigue detection method (CFDM) combined with facial recognition. The results of the ablation experiments show that the attention mechanism and face recognition can effectively improve the accuracy of the fatigue detection model, increasing the model’s AUC by 6.7% and 3.8%, respectively. And even when the face is occluded or the driver wears a mask, the comprehensive model we propose can still correctly detect the fatigue state, and the AUC of the model reaches 0.92, while the existing fatigue detection algorithms based on facial key-points will basically fail.