Towards optimized TBM cutter changing policies with reinforcement learning

Abstract

In tunnel boring machine (TBM) excavation, cutter maintenance is necessary, but the time for it has to be minimized for efficiency. Although there is extensive literature on TBM cutter wear and predictive maintenance for different industrial applications, there is no optimized policy for cutter changes in TBM tunnelling today. This study aims to investigate the application of reinforcement learning (RL) – a branch of machine learning – for finding optimized policies for cutter changing that maximize the number of working cutters and minimize the maintenance effort. A simulation of a TBM excavation process is developed that focuses on the cutter wear and an agent that controls when cutters must be changed. The simulation uses generated parameters that indicate the cutter life, but the results could be transferred to real sensor data in future excavations once that level of development is reached. The article presents the first results from this RL scenario which can give valuable insights into TBM excavation logistics and presents a challenging multiaction-selection RL problem.