Addressing Data Imbalance in Malware Detection: A Comparison Analysis of Machine Learning Models for Malware Detection on Balanced and Imbalanced Datasets

Abstract

Class imbalance is a key issue in developing effective machine learning models for malware detection. In this study, we investigate the impact of class imbalance on model performance based on a malware-majority version of the TUANDROMD dataset, in which 75% of the samples represent malicious programs. We tested the effectiveness of three widely used oversampling strategies-SMOTE, SMOTE-ENN, and ADASYN-to address this imbalance and improve classifier sensitivity to the minority class (goodware). Three machine learning models-Random Forest, Decision Tree, and Support Vector Classifier (SVC)-were trained and tested on imbalanced and balanced datasets. Our results indicate that while imbalanced datasets yield seemingly accurate results, they have poor recall and F1-score for goodware. With the use of resampling techniques, particularly SMOTE-ENN, model performance on the minority class was significantly improved with only a minor decline in overall accuracy. This paper stresses the importance of balancing to not only improve detection fairness but also avoid the danger of false negatives in real-world malware detection systems.