On the evaluation of the homogeneous ensembles with CV-passports

Abstract

Ensembles are often capable of greater prediction accuracy than any of their individual members. As a consequence of the diversity between individual base-learners, an ensemble will not suffer from overfitting. In this regard, development of a systematic and automatic approach for the evaluation of ensemble solutions is particularly important. Based on the mechanism of homogeneous ensembling (known, also, as bagging), we can construct a passport of the solution as a unified validation trajectory against all available training data. Assuming that passports mimic closely the corresponding test solutions, we can use them for the consideration of many tasks including optimizations of blends and ensembles, calculation of the biases and any other tests as required. The reported results were obtained online during the International PAKDD data mining competition in 2010, where we were awarded a certificate for the fourth best result. We, also, report results from the second most popular contest on the Kaggle platform named "Credit", where we demonstrate one of the best results. © Springer-Verlag 2013.