A hadoop based framework integrating machine learning classifiers for anomaly detection in the internet of things

Abstract

In recent years, different variants of the botnet are targeting government, private organi-zations and there is a crucial need to develop a robust framework for securing the IoT (Internet of Things) network. In this paper, a Hadoop based framework is proposed to identify the malicious IoT traffic using a modified Tomek‐link under‐sampling integrated with automated Hyper‐param-eter tuning of machine learning classifiers. The novelty of this paper is to utilize a big data platform for benchmark IoT datasets to minimize computational time. The IoT benchmark datasets are loaded in the Hadoop Distributed File System (HDFS) environment. Three machine learning ap-proaches namely naive Bayes (NB), K‐nearest neighbor (KNN), and support vector machine (SVM) are used for categorizing IoT traffic. Artificial immune network optimization is deployed during cross‐validation to obtain the best classifier parameters. Experimental analysis is performed on the Hadoop platform. The average accuracy of 99% and 90% is obtained for BoT_IoT and ToN_IoT da-tasets. The accuracy difference in ToN‐IoT dataset is due to the huge number of data samples cap-tured at the edge layer and fog layer. However, in BoT‐IoT dataset only 5% of the training and test samples from the complete dataset are considered for experimental analysis as released by the da-taset developers. The overall accuracy is improved by 19% in comparison with state‐of‐the‐art tech-niques. The computational times for the huge datasets are reduced by 3–4 hours through Map Re-duce in HDFS.