Randomized Quasi-Monte Carlo Simulation of Markov Chains with an Ordered State Space

Abstract

We study a randomized quasi-Monte Carlo method for estimating the state distribution at each step of a Markov chain with totally ordered (discrete or continuous) state space. The number of steps in the chain can be random and unbounded. The method simulates n copies of the chain in parallel, using a (d + 1)-dimensional low-discrepancy point set of cardinality n, randomized independently at each step, where d is the number of uniform random numbers required at each transition of the Markov chain. The method can be used in particular to get a low-variance unbiased estimator of the expected total cost up to some random stopping time, when state-dependent costs are paid at each step. We provide numerical illustrations where the variance reduction with respect to standard Monte Carlo is substantial.