Sensor-Driven Surrogate Modeling and Control of Nonlinear Dynamical Systems Using FAE-CAE-LSTM and Deep Reinforcement Learning

Abstract

Highlights: What are the main findings? A surrogate-based deep reinforcement learning (DRL) framework (FAE–CAE–LSTM) is proposed. Achieves higher accuracy compared to DMD, POD, and conventional autoencoders. Provides 5–10× computational speedup, enabling real-time flow control. Incorporates a CNN–MLP reward estimator that enhances policy performance. What is the implication of the main findings? Enables robust modeling of high-dimensional nonlinear flow dynamics. Reduces dependence on full-order PDE solvers, minimizing control delays. Scalable to turbulent and three-dimensional flow control problems. Offers broad applicability in fluid dynamics, energy systems, and engineering control. In cyber-physical systems governed by nonlinear partial differential equations (PDEs), real-time control is often limited by sparse sensor data and high-dimensional system dynamics. Deep reinforcement learning (DRL) has shown promise for controlling such systems, but training DRL agents directly on full-order simulations is computationally intensive. This paper presents a sensor-driven, non-intrusive reduced-order modeling (NIROM) framework called FAE-CAE-LSTM, which combines convolutional and fully connected autoencoders with a long short-term memory (LSTM) network. The model compresses high-dimensional states into a latent space and captures their temporal evolution. A DRL agent is trained entirely in this reduced space, interacting with the surrogate built from sensor-like spatiotemporal measurements, such as pressure and velocity fields. A CNN-MLP reward estimator provides data-driven feedback without requiring access to governing equations. The method is tested on benchmark systems including Burgers’ equation, the Kuramoto–Sivashinsky equation, and flow past a circular cylinder; accuracy is further validated on flow past a square cylinder. Experimental results show that the proposed approach achieves accurate reconstruction, robust control, and significant computational speedup over traditional simulation-based training. These findings confirm the effectiveness of the FAE-CAE-LSTM surrogate in enabling real-time, sensor-informed, scalable DRL-based control of nonlinear dynamical systems.