Goodness of Fit and Parameter Uncertainty

Abstract

After calculating the best-fit values of model parameters, it is necessary to determine whether the model is actually a correct description of the data, even when we use the best possible values for the free parameters. In fact, only when the model is acceptable are best-fit parameters meaningful. The acceptability of a model is typically addressed via the distribution of the χ2 statistic or, in the case of Poisson data, of the Cash statistic. A related problem is the estimate of uncertainty in the best-fit parameters. This chapter describes how to derive confidence intervals for fit parameters, in the general case of Gaussian distributions that require χ2 minimization, and for the case of Poisson data requiring the Cash statistic. We also study whether a linear relationship between two variables is warranted at all, providing a statistical test based on the linear correlation coefficient. This is a question that should be asked of a two-variable dataset prior to any attempt to fit with a linear or more sophisticated model.