A Sandwich Proof of the Shannon-McMillan-Breiman Theorem

Abstract

Let {Xt} be a stationary ergodic process with distribution P admitting densities p(x0,... and xn-1) relative to a reference measure M that is finite order Markov with stationary transition kernel. Let IM(P) denote the relative entropy rate. Then n-1log p(X0,... and Xn-1) ? IM(P) a.s. (P). We present an elementary proof of the Shannon-McMillan-Breiman theorem and the preceding generalization, obviating the need to verify integrability conditions and also covering the case IM(P) = 8. A sandwich argument reduces the proof to direct applications of the ergodic theorem.