Compression of Individual Sequences via Variable-Rate Coding

Abstract

Compressibility of individual sequences by the class of generalized finite-state information-Iossless encoders is investigated. These encoders can operate in a variable-rate mode as well as a fixed-rate one, and they allow for any finite-state scheme of variable-length-to-variable-Iength coding. For every individual infinite sequence x a quantity p (x) is defined, called the compressibility of x, which is shown to be the asymptotically attainable lower bound on the oompression ratio that can be achleved for x by any finite-state encoder. This is demonstrated by means of a constructive coding theorem and its converse that, apart from their asymptotic significance, also provide useful performance criteria for finite and practical data-compression tasks. The proposed concept of compressibility is also shown to play a role analogous to that of entropy in classical information theory where one deals with probabilistic ensembles of sequences rather than with individual sequences. Wbiie the definition of ρ (x) allows a different machine for each different sequence to be compressed, the coustructive coding theorem leads to a universal algorithm that is asymptotically optimal for all sequences. © 1978 IEEE