Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning

Abstract

This paper presents a solution to address the challenges of unexpected events in the operation of metro trains, which can lead to increased delays and safety risks. An improved Q-learning algorithm is proposed to reschedule train timetables via incorporating train detention and different section running times as actions. To enhance computational efficiency and convergence rate, a simulated annealing dynamic factor is introduced to improve action selection strategies. Additionally, importance sampling is employed to evaluate different policies effectively. A case study of Shenzhen Metro is conducted to demonstrate the effectiveness of the proposed method. The results show that the method achieves convergence, fast computation speed, and real-time adjustment capabilities. Compared to traditional methods such as no adjustment, manual adjustment, and FIFO (First-In-First-Out), the proposed method significantly reduces the average total train delay by 54% and leads to more uniform train headways. The proposed method utilizes a limited number of variables for practical state descriptions, making it well suited for real-world applications. It also exhibits good scalability and transferability to other metro systems.