Markov Chain Monte Carlo

Abstract

Markov chains are defined and their relation to transition matrices and kernels are described. Markov chain Monte Carlo (MCMC) employs the Metropolis-Hastings (MH) algorithm as another sampling mechanism. The utility of MCMC is that it can be used to sample complex PDFs, even if the normalization constants are unknown. It is shown that the MH process necessarily leads to the reversibility of the transition function. The myth of burn-in to obtain proper samples is discussed and multidimensional sampling is discussed. The Gibbs sampler is introduced as a special case of the MH algorithm. Then Bayesian probability concepts are introduced along with their prior, posterior and likelihood distributions. With a rigorous foundation, this Bayesian approach is then applied to inference and decision theory. Keywords: Markov chain; Markov chain Monte Carlo (MCMC); transition matrices and kernels; Metropolis-Hastings algorithm; Gibbs sampling; Bayesian analysis; inference and decision making.