On the Uniform Consistency of Bayes Estimates for Multinomial Probabilities

Abstract

A k-sided die is thrown n times, to estimate the probabilities θ1, ..., θk of landing on the various sides. The MLE of θ is the vector of empirical proportions p = (p1, ..., pk). Consider a set of Bayesians that put uniformly positive prior mass on all reasonable subsets of the parameter space. Their posterior distributions will be uniformly concentrated near p. Sharp bounds are given, using entropy. These bounds apply to all sample sequences: There are no exceptional null sets.