Automated assembly sequence planning and scheduling in precast building projects using reinforcement learning

Abstract

Efficient construction of precast concrete buildings (PCB) with high quality and safety requires adequate planning and scheduling of the assembly of precast components (PC). Traditional manual assembly sequence planning and scheduling (ASPS) methods often result in sub-optimal and error-prone schedules, especially in complex projects. Existing metaheuristic-based ASPS methods lack generalizability and capability to handle dynamic conditions in construction projects. Reinforcement Learning (RL) offers a promising solution to these limitations. However, existing RL-based construction scheduling studies overlook project uncertainties and rely on value-based methods, which are computationally inefficient for large-scale problems. This study proposes a novel method for optimal ASPS of PCB projects using RL integrated with Monte-Carlo Sampling (MCS). The method uses a Temporal Graph Network (TGN) and Multi-Layer Perceptron (MLP) to create embeddings of the relevant input information, while the Proximal Policy Optimization (PPO) RL algorithm is employed for generating the assembly schedules using the embeddings. Reward shaping is utilized to generate schedules that simultaneously minimize time, cost, and constraint violations. Preliminary results from a case study on a hypothetical PC project with 34 precast concrete walls demonstrate the proposed method to provide stable learning and generate optimum assembly schedules. The proposed method outperforms existing value-based RL approaches for construction scheduling, demonstrating superior training stability, solution quality, and execution efficiency.