Online PID Tuning Strategy for Hydraulic Servo Control Systems via SAC-Based Deep Reinforcement Learning

Abstract

Proportional–integral–derivative (PID) control is the most common control technique used in hydraulic servo control systems. However, the nonlinearity and uncertainty of the hydraulic system make it challenging for PID control to achieve high-precision control. This paper proposes a novel control strategy that combines the soft actor-critic (SAC) reinforcement learning algorithm with the PID method to address this issue. The proposed control strategy consists of an upper-level controller based on the SAC algorithm and a lower-level controller based on the PID control method. The upper-level controller continuously tunes the control parameters of the lower-level controller based on the tracking error and system status. The lower-level controller performs real-time control for the hydraulic servo system with a control frequency 10 times higher than the upper controllers. Simulation experiments demonstrate that the proposed SAC-PID control strategy can effectively address disturbances and achieve high precision control for hydraulic servo control systems in uncertain working conditions compared with PID and fuzzy PID control methods. Therefore, the proposed control strategy offers a promising approach to improving the tracking performance of hydraulic servo systems.