Reinforcement learning to drive a car by pattern matching

Abstract

This research focuses on vision guided autonomous steering of a wheeled vehicle and tries to implement elementary recognition and learning abilities. While other researchers mainly focussed on using neural networks (e.g. [1], [2], [3], [4]) or explicit modelling of vehicle and environment (e.g. [5],[6],[7]) we established a system which classifies the video information and the vehicle behaviour into patterns and uses a very quick Pattern Matching Algorithm to decide on the required interactions with the environment (i.e. issuance of steering commands) in order to autonomously steer a vehicle Within our research, such capabilities of driving by Pattern Matching got successfully implemented but the quality of the driving behaviour is strongly dependant on knowledge on how to react in certain driving situations. Any feedback on the appropriateness of the driving behaviour is delayed and unspecific in relation to single issued steering commands. Therefore, a further central element in this research is a machine learning algorithm learning by reinforcement based on noisy and delayed rewards. An initial Reinforcement Learning algorithm (e.g. [8], [9]) has been implemented which shows very promising results for creating a system which autonomously steers a car purely based on visual information and even self-improves driving behaviour over time. © Springer-Verlag Berlin Heidelberg 2002.