Reinforcement learning for optimal control of stochastic nonlinear systems

Abstract

A reinforcement learning (RL) approach is developed in this work for nonlinear systems under stochastic uncertainty. A stochastic control Lyapunov function (SCLF) candidate is first constructed using neural networks (NNs) as an approximator to the value function, and then a control policy designed using this SCLF is developed to ensure the stability in probability of the stochastic nonlinear system. An RL algorithm is proposed for stochastic nonlinear systems to iteratively update the value function and control policy, driving them toward optimal values. We further extend its feasibility under the sample-and-hold implementation of control actions, and demonstrate its application to two chemical reactor examples to show its practical advantages and efficiency.