Real-Time Multispectral Pedestrian Detection with a Single-Pass Deep Neural Network

Abstract

The need for fast and robust pedestrian detection in various applications is growing every day. The addition of a thermal camera could help solving this problem resulting in higher detection accuracy in day but especially during night and bad weather conditions. Using convolutional neural networks, the leading technology in the field of object detection and classification, we propose a network architecture and training method for an accurate real-time multispectral pedestrian detector. We select a regression based single-pass network architecture with pre-trained weights from the Pascal VOC 2007 dataset. The network is then transfer-learned without changing the architecture but taking as input three image channels composed from information of the four available image channels (RGB+T). In our experiments we compare the results of different input-channel compositions and select a top performing combination. Our results show that this simple approach easily outperforms the improved ACF+T+THOG detector, coming close to the accuracy of other state-of-the-art multispectral CNNs with a log-average miss-rate of 31.2% measured on the KAIST multispectral benchmark dataset. Our main contribution: it runs as fast as 80FPS, estimated 10 × faster than the closest competitors.