pymcmcstat: A Python Package for Bayesian Inference Using Delayed Rejection Adaptive Metropolis

Abstract

Many scientific problems require calibrating a set of model parameters to fit a set of data. Various approaches exist for performing this calibration, but not all of them account for underlying uncertainty within the problem. Examples of this uncertainty include random noise within experimental measurements as well as errors due to model discrepancy; i.e., missing physics in the model. A Bayesian framework provides a natural perspective from which to perform model calibration to accommodate this uncertainty. To utilize this approach, we make several assumptions regarding the problem. 1. Parameters (q) are treated as random variables with underlying distributions instead of unknown but fixed values. 2. Observations F data (i) are expected to be equal to the model response F (i; q) plus independent and identically distributed random errors ϵ i → F data (i) = F (i; q) + ϵ i where ϵ i ∼ N (0, σ 2). The goal of the calibration process is to infer the parameters' posterior distributions given a set of observations-π(q|F data (i)). Once these parameter distributions are known, the Bayesian approach provides a natural framework in which to consider how this uncertainty propagates and affects model predictions. We designed the pymcmcstat package for engineers and scientists interested in using Bayesian methods to quantify model parameter uncertainty. Furthermore, we had the goal of providing a Python platform for researchers familiar with the MATLAB toolbox mcmcstat developed by Marko Laine. To accommodate a diverse audience, we constructed several tutorials to guide the user through the various stages of setting up a problem, such as defining the data structure, model parameters, and simulation options. Currently, the package is limited to Gaussian likelihood and prior functions; however, these are still suitable for a wide variety of scientific problems. As many individuals are not necessarily familiar with the statistical nomenclature behind the Bayesian approach, the package simply requires the user to define a function that calculates the sum-of-squares error with respect to the model and the observations which is consistent with the second assumption listed above. Information known a priori about the parameter distributions is defined in the prior function; however, the default program behavior is to use a uniform prior which is a common approach for these types of problems.