Model selection and model averaging

Abstract

Model selection methods provide a way to select one model amongst a set of models in a statistically valid way. Such methods include tools for variable selection in regression models. Asymptotic properties such as consistency and efficiency, the specific use of the model, or properties regarding minimization of a certain risk function such as the expected prediction error, may help to decide which method to choose. Model selection is a special case of model averaging where the estimators obtained from different models are combined in a weighted average. Model averaging avoids the selection of one model. The choice of the weights may be determined by a model selection method or may come from a priori knowledge in a Bayesian framework.