Using XGBoost for Cyberattack Detection and Analysis in a Network Log System with ELK Stack

Abstract

Recently, cyberattackers have been developing more sophisticated ways to attack systems. Accordingly, identifying these attacks is getting more complicated in time. On many situations, network administrators were not capable of recognizing these attacks effectively or respond quickly. Whereas, to monitor and analyze the network log data which is very large and complicated is challenging. Therefore, in this case, there is a need to use artificial intelligence and machine learning techniques. In this paper, we develop a monitoring and analysis system for network log data. First, we used Elasticsearch, Logstash, and Kibana (ELK Stack) to monitor the network system. Second, we analyze the network log data use ‘eXtreme Gradient Boosting’ (XGBoost) to build a model of attack event detections. Finally, we use the XGBoost model to do cross-validated with the ELK Stack.