On the optimal coding

Abstract

Novel coding schemes are introduced and relationships between optimal codes and Huffman codes are discussed. It is shown that, for finite source alphabets, the Huffman coding is the optimal coding, and conversely the optimal coding needs not to be the Huffman coding. It is also proven that there always exists the optimal coding for infinite source alphabets. We show that for every random variable with a countable infinite set of outcomes and finite entropy there exists an optimal code constructed from optimal codes for truncated versions of the random variable. And the average code word lengths of any sequence of optimal codes for the truncated versions converge to that of the optimal code. Furthermore, a case study of data compression is given. Comparing with the Huffman coding, the optimal coding is a more flexible compression method used not only for statistical modeling but also for dictionary schemes.