Sequences of Dependent Trials. Markov Chains

Abstract

The chapter opens with in Sect. 13.1 presenting the key definitions and first examples of countable Markov chains. The section also contains the classification of states of the chain. Section 13.2 contains necessary and sufficient conditions for recurrence of states, the Solidarity Theorem for irreducible Markov chains and a theorem on the structure of a periodic Markov chain. Key theorems on random walks on lattices are presented in Sect. 13.3, along with those for a general symmetric random walk on the real line. The ergodic theorem for general countable homogeneous chains is established in Sect. 13.4, along with its special case for finite Markov chains and the Law of Large Numbers and the Central Limit Theorem for the number of visits to a given state. This is followed by a short Sect. 13.5 detailing the behaviour of transition probabilities for reducible chains. The last three sections are devoted to Markov chains with arbitrary state spaces. First the ergodicity of such chains possessing a positive atom is proved in Sect. 13.6, then the concept of Harris Markov chains is introduced and conditions of ergodicity of such chains are established in Sect. 13.7. Finally, the Laws of Large Numbers and the Central Limit Theorem for sums of random variables defined on a Markov chain are obtained in Sect. 13.8.