Ergodic behavior of graph entropy

Abstract

For a positive integer n, let Xn be the vector formed by the first n samples of a stationary ergodic finite alphabet process. The vector Xn is hierarchically represented via a finite rooted acyclic directed graph Gn. Each terminal vertex of Gn carries a label from the process alphabet, and Xn can be reconstituted as the sequence of labels at the ends of the paths from root vertex to terminal vertex in Gn. The entropy H(Gn) of the graph Gn is defined as a nonnegative real number computed in terms of the number of incident edges to each vertex of Gn. An algorithm is given which assigns to Gn a binary codeword from which Gn can be reconstructed, such that the length of the codeword is approximately equal to H(Gn). It is shown that if the number of edges of Gn is o(n), then the sequence (H(Gn)=n) converges almost surely to the entropy of the process. © 1997 American Mathematical Society.