Epsilon Entropy of Stochastic Processes

Abstract

This paper introduces the concept of epsilon-delta entropy for "probabilisticmetric spaces." The concept arises in the study of efficient datatransmission, in other words, in "Data Compression." In a case ofparticular interest, the space is the space of paths of a stochasticprocess, for example L2[ 0, 1] under the probability distributioninduced by a mean-continuous process on the unit interval. For anyepsilon and delta both greater than zero, the epsilon-delta entropyof any probabilistic metric space is finite. However, when deltais zero, the resulting entropy, called simply the epsilon entropyof the space, can be infinite. We give a simple condition on theeigenvalues of a process on L2[ 0, 1] such that any process satisfyingthat condition has finite epsilon entropy for any epsilon greaterthan zero. And, for any set of eigenvalues not satisfying the givencondition, we produce a mean-continuous process on the unit intervalhaving infinite epsilon entropy for every epsilon greater than zero.The condition is merely that â?? nÏ?n 2 be finite, where Ï?1 2 â?¥Ï?2 2 â?¥ â?¯ are the eigenvalues of the process.