DCA-Based Weighted Bagging: A New Ensemble Learning Approach

Abstract

Ensemble learning is a highly efficient method that combines multiple machine learning models to improve the accuracy and robustness of predictions. Bagging is a popular approach of ensemble learning that involves training a group of independent classifiers on various subsets of the training data and combining their predictions using a voting or average technique. In this study, we present a new weighted bagging approach based on difference of convex functions algorithm (DCA), called BaggingDCA. The proposed algorithm combines multiple base models trained on different subsets of the training data and assigns a weight to each model based on its performance. The weights of the ensemble model are determined thanks to LS-DC, a unified approach for various loss functions in machine learning, both convex and non-convex. We evaluated our proposed algorithm on several benchmark datasets and compared its performance to existing bagging methods. We show that the proposed algorithm using both convex and nonconvex losses improves upon standard bagging, and also outperforms dynamic weighting bagging in terms of prediction accuracy.