Information Flow and Entropy Production on Bayesian Networks

Abstract

This chapter presents an introduction to a theoretical framework of information thermodynamics based on Bayesian networks. It reviews the basic properties of information contents: the Shannon entropy, the relative entropy, the mutual information, and the transfer entropy. The chapter describes stochastic thermodynamics by focusing on a simple case of Markovian dynamics. It discusses the concept of entropy production and reviews the basic concepts and terminologies of Bayesian networks. The chapter discusses the general theory of information thermodynamics on Bayesian networks, and derives the generalized second law of thermodynamics, including the transfer entropy. It applies the general theory to special situations such as repeated measurements and feedback control. In particular, the chapter discusses the relationship between approach based on the transfer entropy and another approach based on the dynamic information flow. It summarizes this chapter and discusses the future prospects of information thermodynamics.