Distributed Training of Deep Autoencoder for Network Intrusion Detection

Abstract

The amount of data being exchanged over the internet is enormous. Attackers are finding novel ways to evade rules, investigate network defenses, and launch successful attacks. Intrusion detection is one of the effective means to counter attacks. As the network traffic continues to grow, it can be challenging for network administrators to detect intrusions. In huge networks connected with millions of computers Terabytes/Zettabytes of data is generated every second. Deep Learning is an effective means for analyzing network traffic and detecting intrusions. In this article, distributed autoencoder on the CSE-CIC-IDS2018 dataset is implemented by considering all the classes of the dataset. The proposed work is implemented on Azure Cloud using distributed training as it helps in speeding up the training process, thereby detecting intrusions faster. An overall accuracy of 98.96 % is achieved. By leveraging such parallel computing into the security process, organizations may accomplish operations more quickly and respond to risks and remediate them at a rate that would not be possible with manual human capabilities alone.