Efficient Estimation of Transition Probabilities in a Markov Chain

Abstract

Let {X 1, ⋯, X N } be N observations on an m-state Markov chain with stationary transition probability matrix $\mathbf{P} = (p_{ij}), p_{ij} > 0, i,j = 1,\cdots, m$, where N is a random variable. For any parametric function of P, the information inequality gives a lower bound on the variance of an unbiased estimator; attaining the lower bound depends on whether the sampling plan or stopping rule S, the estimator f = f(X 1,⋯, X N), and the function E(f) = g(P) are "efficient". All "efficient triples" (S, f, g) are characterized for the Markov chain in which p ij and p i'j' (i' ≠ i) are not related functionally. It is also shown that efficient triples do not exist if $m > 2$ and g is a function of two or more rows of P. For the case m = 2, efficient triples in which g's are functions of both rows are characterized.