Shannon entropy: Axiomatic characterization and application

Abstract

We have presented a new axiomatic derivation of Shannon entropy for a discrete probality distribution on the basis of the postulates of additivity and concavity of the entropy function. We have then modified Shannon entropy to take account of observational uncertainty. The modified entropy reduces, in the limiting case, to the form of Shannon differential entropy. As an application, we have derived the expression for classical entropy of statistical mechanics from the quantized form of the entropy. Copyright © 2005 Hindawi Publishing Corporation.