Railway infrastructure maintenance efficiency improvement using deep reinforcement learning integrated with digital twin based on track geometry and component defects

Abstract

Railway maintenance is a complex and complicated task in the railway industry due to the number of its components and relationships. Ineffective railway maintenance results in excess cost, defective railway structure and components, longer possession time, poorer safety, and lower passenger comfort. Of the three main maintenance approaches, predictive maintenance is the trendy one, and is proven that it provides the highest efficiency. However, the implementation of predictive maintenance for the railway industry cannot be done without an efficient tool. Normally, railway maintenance is corrective when some things fail or preventive when maintenance is routine. A novel approach using an integration between deep reinforcement learning and digital twin is proposed in this study to improve the efficiency of railway maintenance which other techniques such as supervised and unsupervised learning cannot provide. In the study, Advantage Actor Critic (A2C) is used to develop a reinforcement learning model and agent to fulfill the need of the study. Real-world field data over four years and 30 km. is obtained and applied for developing the reinforcement learning model. Track geometry parameters, railway component defects, and maintenance activities are used as parameters to develop the reinforcement learning model. Rewards (or penalties) are calculated based on maintenance costs and occurring defects. The new breakthrough exhibits that using reinforcement learning integrated with digital twin can reduce maintenance activities by 21% and reduce the occurring defects by 68%. Novelties of the study are the use of A2C which is faster and provides better results than other traditional techniques such as Deep Q-learning (DQN), each track geometry parameter is considered without combining into a track quality index, filed data are used to develop the reinforcement learning model, and seven independent actions are included in the reinforcement learning model. This study is the world’s first to contribute a new guideline for applying reinforcement learning and digital twins to improve the efficiency of railway maintenance, reduce the number of defects, reduce the maintenance cost, reduce the possession time for railway maintenance, improve the overall safety of the railway operation, and improve the passenger comfort which can be seen from its results.