Driving control with deep and reinforcement learning in the open racing car simulator

Abstract

Vision-based control is a hot topic in the field of computational intelligence. Especially the development of deep learning (DL) and reinforcement learning (RL) provides effective tools to this field. DL is capable of extracting useful information from images, and RL can learn an optimal controller through interactions with environment. With the aid of these techniques, we consider to design a vision-based robot to play The Open Racing Car Simulator. The system uses DL to train a convolutional neural network to perceive driving data from images of first-person view. These perceived data, together with the car’s speed, are input into a RL-learned controller to get driving commands. In the end, the system shows promising performance.