What Bayes has to Say about the Evidence Procedure

Abstract

The ``evidence'' procedure for settinghyperparameters is essentially the same as thetechniques of ML-II and generalized maximumlikelihood. Unlike those older techniques however,the evidence procedure has been justified (andused) as an approximation to the hierarchicalBayesian calculation. We use several examples toexplore the validity of this justification. Then wederive upper and (often large) lower bounds on thedifference between the evidence procedure's answerand the hierarchical Bayesian answer, for manydifferent quantities. We also touch on subjectslike the close relationship between the evidenceprocedure and maximum likelihood, and theself-consistency of deriving priors by``first-principles'' arguments that don't set thevalues of hyperparameters.