Finite approximation error-based value iteration ADP

Abstract

In this chapter, iterative adaptive dynamic programming (ADP) algorithms are developed to solve optimal control problems for infinite horizon discrete-time nonlinear systems with finite approximation errors. idea is to use iterative ADP algorithms to obtain the iterative control laws that guarantee the iterative value functions to reach the optimums. Then, the numerical optimal control problems are solved by an adaptive learning control scheme based on ADP algorithm. Stability properties of the system under the numerical iterative controls are proved which allow the present iterative ADP algorithm to be implemented both online and offline. Moreover, a general value iteration (GVI) algorithm with finite approximation errors is developed to guarantee the iterative value function to converge to the solution of the Bellman equation. The GVI algorithm permits an arbitrary positive semidefinite function to initialize it, which overcomes the disadvantage of traditional value iteration algorithms. Simulation examples are also included to demonstrate the effectiveness of the present control strategies.