Entropy in Hydrology

Abstract

Although the concept of thermodynamic entropy due to Clausius dates back to the early 1850s, the mathematical theory of informational entropy was not developed until the pioneering work of Shannon in 1948, the development of principle of maximum entropy (POME) and theorem of concentration by Jaynes in 1957, principle of minimum cross entropy by Kullback and Leibler in 1959, and the formulation of entropy in frequency domain by Burg in 1967. The concept of informational entropy is more intuitive, because it is a measure of information or uncertainty which is encountered in daily life. Hence, its application is ubiquitous. If we peruse hydrologic problems, it becomes clear that their solutions involve either measurement of information through data collection, or extraction of information through data analysis, or maximization or minimization of information by optimization, or prediction of information through modeling, or analysis and synthesis of information by simulation, or weighing of information for decision making. Thus, solutions of hydrologic problems may involve the direct application of entropy, examples of which are monitoring network evaluation and design, water resources allocation, and model selection. Solutions of some problems involve the application of the POME, such as derivation of frequency distributions and parameter estimation, whereas solutions of other problems may involve the POME and a flux‐concentration type relation, such as modeling of hydrologic processes. There seems hardly any area in hydrology where entropy cannot be gainfully applied. This paper discusses basic ingredients for the application of entropy theory.Information and uncertainty are part of our daily life. We gather information from data which is measured or collected. We gather the notion of uncertainty from experience based on data. Information and uncertainty are like two sides of the same coin. That is, more the information less the uncertainty. The theory of entropy, discussed in this paper, enables to quantify information and in turn uncertainty. In hydrology, we model a variety of processes and modeling may include prediction of characteristics of these processes, estimation of parameters characterizing the processes, and determination of the probability distribution of hydrologic variables. The modeling can be accomplished using the theory of entropy. This paper discusses applications of the theory of entropy in hydrology. Hydrologic problems can be solved using either the direct application of entropy, principle of maximum entropy, principle of minimum cross entropy, or theorem of concentration Decision making involves either weighing of information, minimization or maximization of information, extraction of information or measurement of information which is characterized by the theory of entropy Entropy theory can be applied for the derivation of probability distributions or derivation of physical relationships and estimation of parameters thereof