Randomized approximation of the MDL for stochastic models with hidden variables

Abstract

A randomized algorithm minimum description length (RAMDL), which efficiently approximates the stochastic complexity (SC) for stochastic models with hidden variables is proposed. The objectives of RAMDL are to decompose the SC into a sequence of segments and to sequentially apply the Markov chain Monte Carlo method to approximate each segment. RAMDL is analyzed by giving upper bounds on its approximation error to the SC. The bounds are obtained as functions of the number of random samplings that RAMDL makes and the complexity of the hypothesis class. The tradeoff relation between the statistical approximation accuracy and computational complexity were sized up in general form.