An elementary derivation of the large deviation rate function for finite state markov chains

Abstract

Large deviation theory is a branch of probability theory that is devoted to a study of the "rate" at which empirical estimates of various quantities converge to their true values. The object of study in this paper is the rate at which estimates of the doublet frequencies of a Markov chain over a finite alphabet converge to their true values. In the case where the Markov process is actually an independent and identically distributed (i.i.d.) process, the rate function turns out to be the relative entropy (or Kullback-Leibler divergence) between the true and the estimated probability vectors. This result is a special case of a very general result known as Sanov's theorem and dates back to 1957. Moreover, since the introduction of the "method of types" by Csiszar and his co-workers during the 1980s, the Proof of this version of Sanov's theorem has been "elementary," using some combinatorial arguments. However, when the i.i.d. process is replaced by a Markov process, the available Proofs are far more complex. The main objective of this paper is therefore to present a first-principles derivation of the LDP for finite state Markov chains, using only simple combinatorial arguments (e.g. the method of types), thus gathering in one place various arguments and estimates that are scattered over the literature. The approach presented here extends naturally to multi-step Markov chains. © 2013 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society.