Automatic Tracking Control Strategy of Autonomous Trains Considering Speed Restrictions: Using the Improved Offline Deep Reinforcement Learning Method

Abstract

Previous research on automatic control of high-speed trains in speed limit sections is insufficient. This article proposes a new offline reinforcement learning strategy for automatic tracking of autonomous trains. Firstly, the operating speed and deceleration starting point were determined for different speed limit scenarios. Then, a tracking controller based on the improved offline conservative Q-learning (CQL) algorithm was designed to avoid frequent interaction between the train and the environment. Selected an appropriate policy to implement the CQL algorithm. The data samples were reclassified to increase sample concentration. The value and strategy network structure was redesigned. The state space and action space of tracking trains were limited, and the dimension of state space was increased. A multi-objective reward function was designed to distinguish the tracking process of trains in different sections. The simulation results show that the proposed high-speed railway tracking interval automatic control algorithm is superior to traditional online reinforcement learning methods in terms of safety, comfort, and convergence efficiency.