A Convexity Property in the Theory of Random Variables Defined on a Finite Markov Chain

Abstract

Let P = (pjk) be the transition matrix of an ergodic, finite Markov chain with no cyclically moving sub-classes. For each possible transition (j, k), let Hjk(x) be a distribution function admitting a moment generating function fjk(t) in an interval surrounding t = 0. The matrix P(t) = {pjkfjk(t)} is of interest in the study of the random variable Sn = X1 + ⋯ + Xn, where Xm has the distribution Hjk(x) if the mth transition takes the chain from state j to state k. The matrix P(t) is non-negative and therefore possesses a maximal positive eigenvalue α1(t), which is shown to be a convex function of t. As an application of the convexity property, we obtain an asymptotic expression for the probability of tail values of the sum Sn, in the case where the Xm are integral random variables. The results are related to those of Blackwell and Hodges [1], whose methods are followed closely in Section 5, and Volkov [4], [5], who treats in detail the case of integer-valued functions of the state of the chain, i.e., the case fjk(t) = exp(βkt) (βk integral). CR - Copyright © 1961 Institute of Mathematical Statistics