Formalization of finite-state discrete-time Markov chains in HOL

Abstract

The mathematical concept of Markov chains is widely used to model and analyze many engineering and scientific problems. Markovian models are usually analyzed using computer simulation, and more recently using probabilistic model-checking but these methods either do not guarantee accurate analysis or are not scalable. As an alternative, we propose to use higher-order-logic theorem proving to reason about properties of systems that can be described as Markov chains. As the first step towards this goal, this paper presents a formalization of time homogeneous finite-state Discrete-time Markov chains and the formal verification of some of their fundamental properties, such as Joint probabilities, Chapman-Kolmogorov equation and steady state probabilities, using the HOL theorem prover. For illustration purposes, we utilize our formalization to analyze a simplified binary communication channel. © 2011 Springer-Verlag.