Parallel processing using big data and machine learning techniques for intrusion detection

Abstract

Currently, information technology is used in all the life domains. Many devices and equipment produce data and transfer them across the network. These transfers are not always secured and can contain new menaces and attacks invisible by the current security tools. Moreover, the large amount and variety of the exchanged data make the identification of the intrusions more difficult in terms of detection time. To solve these issues, we suggest in this paper, a new approach based on storing the large amount and variety of network traffic data employing big data techniques, and analyzing these data with machine learning algorithms, in a distributed and parallel way, in order to detect new hidden intrusions with less processing time. According to the results of the experiments, the detection accuracy of the machine learning methods reaches up to 99.9%, and their processing time has been reduced considerably by applying them in a parallel and distributed way, which proves that our proposed model is very effective for the detection of new hidden intrusions.