Continuous-time controlled jump Markov processes on the finite horizon

Abstract

We study continuous-time controlled Markov chains on the finite horizon. For the Markov decision problem, we show that the value function is the unique solution of the corresponding dynamic programming equation. This leads to the existence of an optimal Markov control. We then consider a zero-sum game. We show that the value function exists and is the unique solution of the corresponding Isaacs equations. This yields the existence of a pair of saddle point Markov strategies.