Digital Twin With Soft Actor-Critic Reinforcement Learning for Transitioning From Industry 4.0 to 5.0

Abstract

Industry 5.0 builds on the foundations of Industry 4.0 and ensures the transition to a sustainable, flexible, and human-centric workplace. The transition from Industry 4.0 to Industry 5.0 addresses important limitations of traditional automation, such as limited flexibility, human-machine collaboration. Industry 5.0 seeks to overcome these issues through evolution, with an emphasis on sustainability, resilience, and human-centric interactions. However, these changes come with challenges related to real-time adaptation and optimal resource allocation in dynamic industrial environments. To facilitate this transition, we propose a Digital Twin (DT) system that employs a Soft Actor-Critic (SAC), which is a reinforcement learning algorithm. This integration enhances adaptability in real-time, efficient resource allocation, and flexibility in dynamic industrial environments. The SAC-DT algorithm is evaluated using three main scenarios: demand change management, real-time control under uncertain conditions, and optimization of performance. These scenarios demonstrate how the SAC-DT increases productivity, adaptability, reliability, energy efficiency, and reduced downtime. The proposed scheme is compared with baseline models which include DRL-DRP and GCNN-DRL. The analyses show that the SAC-DT scheme yields better productivity, flexibility and, resource utilization efficiency, thus highlighting the applicability of this proposed scheme as valuable to move closer to Industry 5.0 that will guarantee solid, reliable, pragmatic, sustainable technological implementations that meet industrial goals and requirements in future.