Combining asynchronous events and traditional frames for steering angle prediction

Abstract

Advances in deep learning over the last decade enabled by the availability of more computing resources have revived interest in end-to-end neural network methods for command prediction in vehicle control. Most of the existing frameworks in the literature make use of visual data from conventional video cameras to infer low level (steering wheel, speed, etc.) or high level (curvature, driving path and more) commands for actuation. In this paper, we propose an efficient convolutional neural network model that takes both perceptual data in the form of signals (events) from an event-based sensor and traditional frames from the same camera to generate steering wheel angle. We show that our model outperforms many state-of-the-art deep learning approaches using just one type of input among regular frames or events while being much more efficient.