The I-Measure

Abstract

In Chapter 2, we have shown the relationship between Shannon's infonna-tion measures for two random variables by the diagram in Figure 2.2. For convenience, Figure 2.2 is reproduced in Figure 6.1 with the random variables X and Y replaced by Xl and X2, respectively. This diagram suggests that Shannon's information measures for any n :2: 2 random variables may have a set-theoretic structure. In this chapter, we develop a theory which establishes a one-to-one correspondence between Shannon's information measure s and set theory in full generality. With this correspondence, manipulations of Shannon's infonna-tion measures can be viewed as set operations, thus allowing the rich suite of tools in set theory to be used in information theory. Moreover, the structure of Shannon's information measures can easily be visualized by means of an Figure 6.1. Relationship between entropies and mutual informati on for two random variables. 95 R. W. Yeung, A First Course in Information Theory