The Method of Types

Abstract

The method of types allows one to obtain in an elementary way, in the discrete case, two fundamental results of mathematical information theory, which in the continuous case require a rather formidable technical equipment: Sanov’s principle and the maximum entropy principle. The objects considered by this method are empirical averages, and the basic mathematical notion is that of divergence.