Non-sufficient memories that are sufficient for prediction

Abstract

The causal states of computational mechanics define the minimal sufficient (prescient) memory for a given stationary stochastic process. They induce the ε-machine which is a hidden Markov model (HMM) generating the process. The ε-machine is, however, not the minimal generative HMM and minimal internal state entropy of a generative HMMis a tighter upper bound for excess entropy than provided by statistical complexity. We propose a notion of prediction that does not require sufficiency. The corresponding models can be substantially smaller than the ε-machine and are closely related to generative HMMs. © 2009 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.