Deep Learning from Imperfectly Labeled Malware Data

Abstract

Deep learning approaches have achieved remarkable performance in malware classification and detection. However, their success relies on the availability of large, accurately labeled datasets: a critical yet challenging requirement in the malware domain. In practice, most malware datasets are automatically labeled using outputs from antivirus engines, a process that often introduces significant label noise. Such imperfections can severely degrade the performance and generalizability of deep learning models. To address this challenge, we introduce SLB, a framework designed to robustly train deep learning-based malware systems while simultaneously refining dataset labels. SLB begins by partitioning the dataset into two subsets: a clean set containing samples with reliable labels, and a noisy set with samples that may be mislabeled, to which pseudo labels are assigned. As training progresses, SLB continuously monitors the model's predictions to dynamically update both sets. Specifically, samples in the noisy set that consistently receive predictions aligning with their (observed or pseudo) labels are promoted to the clean set, whereas samples in the clean set that exhibit unstable predictions are reclassified as noisy. This iterative process not only enhances model performance but also progressively corrects labeling errors. We evaluated SLB on multiple security datasets with both synthetic and real-world label noise across various deep learning architectures and ML algorithms. Experimental results show that SLB significantly improves malware detection performance and reduces overall noise. For example, on the Android binary dataset with 25% injected label noise, SLB reduced the noise to below 1.5% while increasing the macro F1 score from 74.51% to 96.03% and the accuracy score from 87.66% to 98.68%.