Cluster analysis, model selection, and prior distributions on models

Abstract

Clustering is an important and challenging statistical problem for which there is an extensive literature. Modeling approaches include mixture models and product partition models. Here we develop a product partition model and a Bayesian model selection procedure based on Bayes factors from intrinsic priors. We also find that the choice of the prior on model space is of utmost importance, almost overshadowing the other parts of the clustering problem, and we examine the behavior of the model posterior probabilities based on different model space priors. We find, somewhat surprisingly, that procedures based on the often-used uniform prior (in which all models are given the same prior probability) lead to inconsistent model selection procedures. We examine other priors, and find that the Ewens-Pitman prior and a new prior, the hierarchical uniform prior, lead to consistent model selection procedures and have other desirable properties. Lastly, we compare the procedures on a range of examples.