A Bayesian approach to classification accuracy inference

Abstract

Bayesian accuracy assessments draw inference about random (super-population) parameters characterizing the classification process and accuracy statistics derived from these parameters. A conventional frequentist approach seeks to estimate the same parameters, but view them as fixed finite population quantities. Both approaches are detailed and contrasted with a real land cover data example. Bayesian results are given for non-informative and informative priors. The latter is justified in past experience. Results from simple and stratified random samplings on overall and class-specific accuracies and kappa coefficients of agreement are detailed for samples representing the 10 per cent, the 50 per cent and the 90 per cent quantile in a Monte Carlo sampling distribution of overall accuracy. A Bayesian approach is recommended for applications with small sample sizes and for quality assurance monitoring where prior data can boost effective sample sizes.© Institute of Chartered Foresters, 2009.