An Improved Binary Simulated Annealing Algorithm and TPE-FL-LightGBM for Fast Network Intrusion Detection

Abstract

With the rapid proliferation of the Internet, network security issues that threaten users have become increasingly severe, despite the widespread benefits of Internet access. Most existing intrusion detection systems (IDS) suffer from suboptimal performance due to data imbalance and feature redundancy, while also facing high computational complexity in areas such as feature selection and optimization. To address these challenges, this study proposes a novel network intrusion detection method based on an improved binary simulated annealing algorithm (IBSA) and TPE-FL-LightGBM. First, by integrating Focal Loss into the loss function of the LightGBM classifier, we introduce cost-sensitive learning, which effectively mitigates the impact of class imbalance on model performance and enhances the model’s ability to learn difficult-to-classify samples. Next, significant improvements are made to the simulated annealing algorithm, including adaptive adjustments of the initial temperature and Metropolis criterion, the incorporation of multi-neighborhood search strategies, and the integration of an S-shaped transfer function. These improvements enable the IBSA method to achieve efficient optimal feature selection with fewer iterations. Finally, the Tree-structured Parzen Estimator (TPE) algorithm is employed to optimize the structure of the FL-LightGBM classifier, further enhancing its performance. Through comprehensive visual analysis, ablation studies, and comparative experiments on the NSL-KDD and UNSW-NB15 datasets, the reliability of the proposed network intrusion detection method is validated.