Neuro-Dynamic Programming: An Overview and Recent Results

Abstract

Neuro-dynamic programming is a methodology for sequential decision making under uncertainty, which is based on dynamic programming.The key idea is to use a scoring function to select decisions in complex dynamic systems, arising in a broad variety of applicationsfrom engineering design, operations research, resource allocation, finance, etc. This is much like what is done in computerchess, where positions are evaluated by means of a scoring function and the move that leads to the position with the bestscore is chosen. Neuro-dynamic programming provides a class of systematic methods for computing appropriate scoring functionsusing approximation schemes and simulation/evaluation of the system’s performance.