The entropy of a discrete real variable

Abstract

The discrete Shannon entropy H was formulated only to measure indeterminacy effected through a set of probabilities, but the indeterminacy in a real-valued discrete variable depends on both the allowed outcomes x and the corresponding probabilities p. A fundamental measure that is sensitive to both x and p is derived here from the total differential entropy of a continuous real variable and its conjugate in the discrete limit, where the conjugate is universally eliminated. The asymptotic differential entropy recovers H plus the new measure, named Ξ, which provides a novel probe of intrinsic organization in sequences of real numbers. © 2012 by the author.