Effect of Balancing Data Using Synthetic Data on the Performance of Machine Learning Classifiers for Intrusion Detection in Computer Networks

Abstract

Attacks on computer networks have increased significantly in recent days, due in part to the availability of sophisticated tools for launching such attacks as well as the thriving underground cyber-crime economy to support it. Over the past several years, researchers in academia and industry used machine learning (ML) techniques to design and implement Intrusion Detection Systems (IDSes) for computer networks. Many of these researchers used datasets collected by various organizations to train ML classifiers for detecting intrusions. In many of the datasets used in training ML classifiers in such systems, data are imbalanced (i.e., not all classes had equal number of samples). ML classifiers trained with such imbalanced datasets may produce unsatisfactory results. Traditionally, researchers used over-sampling and under-sampling for balancing data in datasets to overcome this problem. In this work, in addition to random over-sampling, we also used a synthetic data generation method, called Conditional Generative Adversarial Network (CTGAN), to balance data and study their effect on the performance of various widely used ML classifiers. To the best of our knowledge, no one else has used CTGAN to generate synthetic samples to balance intrusion detection datasets. Based on extensive experiments using widely used datasets NSL-KDD and UNSW-NB15, we found that training ML classifiers on datasets balanced with synthetic samples generated by CTGAN increased their prediction accuracy by up to 8% and improved their MCC score by up to 13%, compared to training the same ML classifiers over imbalanced datasets. We also show that this approach consistently performs better than some of the recently proposed state-of-the-art IDSes on both datasets. Our experiments also demonstrate that the accuracy of some ML classifiers trained over datasets balanced with random over-sampling decline compared to the same ML classifiers trained over original imbalanced dataset.