Compression, Tests for Randomness and Estimating the Statistical Model of an Individual Sequence

Abstract

In this survey we consider finite alphabet stationary sources, where the only available statistics about the source is an individual sequence which is emitted by it. Based on this empirically observed statistics we want to perform a variety of tasks: testing for randomness, estimating the number of free parameters of the unknown statistical model such as the order of a finite Markov source, the number of states of a finite-state-stationary source, etc. We derive universal, asymptotically optimal algorithms, satisfying a Neyman-Pearson like criterion. These algorithms are shown to be related to the Lempel-Ziv data compression algorithm for individual sequences.