Precise approximations of the probability distribution of a Markov process in time: An application to probabilistic invariance

Abstract

The goal of this work is to formally abstract a Markov process evolving over a general state space as a finite-state Markov chain, with the objective of precisely approximating the state probability distribution of the Markov process in time. The approach uses a partition of the state space and is based on the computation of the average transition probability between partition sets. In the case of unbounded state spaces, a procedure for precisely truncating the state space within a compact set is provided, together with an error bound that depends on the asymptotic properties of the transition kernel of the Markov process. In the case of compact state spaces, the work provides error bounds that depend on the diameters of the partitions, and as such the errors can be tuned. The method is applied to the problem of computing probabilistic invariance of the model under study, and the result is compared to an alternative approach in the literature. © 2014 Springer-Verlag.