Adjoint-based optimal control of the expected exit time for stochastic hybrid systems

Abstract

In this paper, we study the problem of controlling the expected exit time from a region for a class of stochastic hybrid systems. That is, we find the least costly feedback control for a stochastic hybrid system that can keep its state inside a prescribed region for at least an expected amount of time. The stochastic hybrid systems considered are quite general: the continuous dynamics are governed by stochastic differential equations, and the discrete mode evolves according to a continuous time Markov chain. Instead of adopting the usual Hamilton-Jacobi viewpoint, we study the problem directly by formulating it as a PDE constrained optimization problem, and propose a solution using adjoint-based gradient descent methods. Numerical results of the proposed approach are presented for several representative examples, and, for the simple case, compared with analytical results. © Springer-Verlag Berlin Heidelberg 2005.