Func-Bagging: An Ensemble Learning Strategy for Improving the Performance of Heterogeneous Anomaly Detection Models

Abstract

Featured Application: This work presents an adaptive weight distribution strategy for bagging-based heterogeneous ensemble learning, with a particular focus on improving classification performance in imbalanced datasets. The proposed method can be applied to tasks such as anomaly detection, where class imbalance is common, and it offers a robust solution for enhancing model accuracy and stability in real-world applications, particularly in domains like fall detection, fault diagnosis, and action identification. In the field of ensemble learning, bagging and stacking are two widely used ensemble strategies. Bagging enhances model robustness through repeated sampling and weighted averaging of homogeneous classifiers, while stacking improves classification performance by integrating multiple models using meta-learning strategies, taking advantage of the diversity of heterogeneous classifiers. However, the fixed weight distribution strategy in traditional bagging methods often has limitations when handling complex or imbalanced datasets. This paper combines the concept of heterogeneous classifier integration in stacking with the weighted averaging strategy of bagging, proposing a new adaptive weight distribution approach to enhance bagging’s performance in heterogeneous ensemble settings. Specifically, we propose three weight generation functions with “high at both ends, low in the middle” curve shapes and demonstrate the superiority of this strategy over fixed weight methods on two datasets. Additionally, we design a specialized neural network, and by training it adequately, validate the rationality of the proposed adaptive weight distribution strategy, further improving the model’s robustness. The above methods are collectively called func-bagging. Experimental results show that func-bagging has an average 1.810% improvement in extreme performance compared to the base classifier, and is superior to stacking and bagging methods. It also has better dataset adaptability and interpretability than stacking and bagging. Therefore, func-bagging is particularly effective in scenarios with class imbalance and is applicable to classification tasks with imbalanced classes, such as anomaly detection.