Quasi-Monte Carlo Methods for Estimating Transient Measures of Discrete Time Markov Chains

Abstract

We describe a new method for the transient simulation of discrete time Markov chains, It is a quasi-Monte Carlo method where different paths axe simulated in parallel, but reordered at each step. We prove the convergence of the method, when the number of simulated paths increases. Using some numerical experiments, we illustrate that the error of the new algorithm is smaller than the error of standard Monte Carlo algorithms. Finally, we propose to analyze continuous time Markov chains by transforming them into a discrete time problem by using the uniformization technique.