Identification entropy

Abstract

Shannon (1948) has shown that a source (U, P, U) with output U satisfying Prob (U = u) = Pu; can be encoded in a prefix code C = {cu : u ε U} ⊂ {0, 1}*such that for the entropy H(P) = Σ -Pulog pu < Σ pucu HI(P) and thus also uεu that L(P) = min max Lc(P,u) > HI(P) C uεu and related upper bounds, which demonstrate the operational significance of identification entropy in noiseless source coding similar as Shannon entropy does in noiseless data compression. Also other averages such as L̄c (P) = 1-u Σ Lc (P, u) are discussed in uεu particular for Huffman codes where classically equivalent Huffman codes may now be different. We also show that prefix codes, where the codewords correspond to the leaves in a regular binary tree, are universally good for this average. © Springer-Verlag Berlin Heidelberg 2006.