A hitchhiker's guide to information theoretical measures in psychology

Abstract

In psychology, as in other sciences, information theory can be used as a tool to complement more standard regression-based methods of data analysis. It is important to see the potential of information theoretical measures as statistical tools without implying a connection to their origins in communication theory and engineering. The use of these measures may provide us with additional insights due to their sensitivity to non-linear relationships, their flexibility to the mixing of data types, and their more straightforward generalization towards investigating higher-order interactions. We briefly reintroduce information theory and compare several measures such as mutual information and co-information with correlation and regression-based methods for the investigation of variable dependence.