Comparative Analysis of Intrusion Detection Models using Big Data Analytics and Machine Learning Techniques

Abstract

Traditional cyber security measures are becoming less effective, leading to rise in modern attacks. However, the ability to analyze and use massive volume of data (big data) to train anomaly based systems that can learn from experience, classify attacks and make decisions can improve prediction of attacks before they actually occur. In this study, to ensure availability, integrity, and confidentiality of information systems, predictive models for intrusion detection that use Big Data and Machine Learning (ML) algorithms were proposed. The proposed approach used a big dataset (CIC-Bell-IDS2017) to independently train three ML classifiers before and after feature selection. Big data analytics tool was also employed for feature scaling and selection in order to normalize data and select the most relevant set of features. Performance evaluation and comparative analysis were done and the results showed there were improvements in the models’ prediction accuracies.