Tutorial 5. Combining estimators to improve performance

Abstract

Despite the diverse pedigrees of Data Mining methods, the underlying algorithms fall into a handful of families, whose properties suggest their likely performance on a given dataset. One typically selects an algorithm by matching its strengths to the properties of one's data. Yet, performance surprises, where competing models rank differently than expected, are common; model inference, even when semi-Automated, seems to yet be as much art as science. Recently however, researchers in several fields have discovered that a simple technique - combining competing models -Almost always improves classification accuracy. (Such "bundling" is a natural outgrowth of Data Mining, since much of the model search process is automated, and candidate models abound.) This tutorial will describe an interdisciplinary collection of powerful model combination methods - including bundling, bagging, boosting, and Bayesian model averaging -And briefly demonstrate their positive effects on scientific, medical, and marketing case studies. The instructors will show why this simple, new idea will often improve a model's accuracy and stability (robustness).