Entropy Optimization Principles and Their Applications

Abstract

A state-of-the-art description of the theory and applications of the various entropy optimization principles is given. These principles include Jaynes' maximum entropy principle (MaxEnt), Kullback's minimum cross-entropy principle (MinxEnt), generalised maximum entropy and minimum cross-entropy principles, inverse entropy optimization principles, minimum interdependence principle, minimax entropy principle and finally, the dual entropy optimization principles. The relation between information-theoretic entropy and thermodynamic entropy is specially recalled in the context of the more general relationship that exist between what are designated as primary and secondary entropies.