Evolution of Ideas on Entropy

Abstract

Our stated goal is to develop a general theory of thermodynamics that we may apply to stochastic processes, but what is thermodynamics? At the mathematical level, thermodynamics is the calculus of entropy. The inequality of the second law forms the starting point for a large number of mathematical relationships that are written among the set of primary variables, entropy, energy, volume and number of particles, and a number of defined functions based on the primary set. The mathematical framework of classical thermodynamics is established as soon as the second law is formulated in mathematical form. The subsequent development of statistical mechanics left this framework intact, while making the connection to the microscopic structure of matter. Even as the basic mathematical framework did not change, the revolution lead by Gibbs introduced a statistical view of entropy that opened an entirely new viewpoint. Starting with Shannon’s work on information theory and Jaynes’s formulation of maximum entropy has now escaped from the realm of physics to invade other fields. In this chapter we review the evolution of ideas on entropy, from the classical view, to Gibbs, Shannon, and Jaynes, and cover the background that gives rise to our development in the chapters that follow.