OPTCON3: An Active Learning Control Algorithm for Nonlinear Quadratic Stochastic Problems

Abstract

In this paper, we describe the new OPTCON3 algorithm, which serves to determine approximately optimal policies for stochastic control problems with a quadratic objective function and nonlinear dynamic models. It includes active learning and the dual effect of optimizing policies, whereby optimal policies are used to learn about the stochastics of the dynamic system in addition to their immediate effect on the performance of the system. The OPTCON3 algorithm approximates the nonlinear model with a time-varying linear model and applies a procedure similar to that of Kendrick to the series of linearized models to calculate approximately optimal policies. The results for two simple economic models serve to test the OPTCON3 algorithm and compare it to previous solutions of the stochastic control problem. Initial evaluations show that the OPTCON3 approach may be promising to enhance our understanding of the adaptive economic policy problem under uncertainty.