The fundamental thermodynamic bounds on finite models

Abstract

The minimum heat cost of computation is subject to bounds arising from Landauer's principle. Here, I derive bounds on finite modeling - the production or anticipation of patterns (time-series data) - by devices that model the pattern in a piecewise manner and are equipped with a finite amount of memory. When producing a pattern, I show that the minimum dissipation is proportional to the information in the model's memory about the pattern's history that never manifests in the device's future behavior and must be expunged from memory. I provide a general construction of a model that allows this dissipation to be reduced to zero. By also considering devices that consume or effect arbitrary changes on a pattern, I discuss how these finite models can form an information reservoir framework consistent with the second law of thermodynamics.