On the combinatorial representation of information

Abstract

Kolmogorov introduced a combinatorial measure of the information I(x : y) about the unknown value of a variable y conveyed by an input variable x taking a given value x. The paper extends this definition of information to a more general setting where 'x = x' may provide a vaguer description of the possible value of y. As an application, the space P({0, 1}n) of classes of binary functions f : [n] → {0, 1}, [n] = {1,..., n], is considered where y represents an unknown function t ∈ {0, 1}[n] and as input, two extreme cases are considered: x = xMd and x = xM′d which indicate that t is an element of a set G ⊆ {0, 1}n that satisfies a property Md or M′d respectively. Property Md (or M′d) means that there exists an E ⊆ [n], |E| = d, such that |trE(G)| = 1 (or 2d) where trE(G) denotes the trace of G on E. Estimates of the information value I(xMd : t) and I(xM′d: t) are obtained. When d is fixed, it is shown that I(xMd : t) ≈ d and I(x M′d : t) ≈ 1 as n → ∞. © Springer-Verlag Berlin Heidelberg 2006.