Large-scale threat traffic analysis and IDS development using software

Abstract

Background/Objectives: Today, modern society is constantly changing with the development of innovations. This development is the leading to the fourth industrial revolution in the fields of AI, cloud and Big data. However, a hyper-connected society is vulnerable to cyber-attacks. However, existing countermeasures have not kept in pace with the rapid development in cyber threats. It is necessary to determine critical and dangerous elements in massive data. This requires new networking technology that can malicious traffic. Methods/Statistical analysis: To address the aforementioned issues, we developed Bro-IDS, which can monitor large-scale traffic for cyber threats. Bro-IDS is an intrusion detection system based on open source software. Bro-IDS also can create network traffic for various logs and be used for traffic measurement or forensic purposes. We perform experiments to analyze large-scale threats in real-time using the ELK Stack (Elastic search, Log stash, Kibana) to validate Bro-IDS. Findings: We performed a cyber threat analysis based using our security monitoring system from July 25th to August 5th 2018 on KREONET. During this period, all connections generated approximately 610 million logs and the total payload collected was approximately 300GB.First, weird logs and notice logs were generated. Weird logs are generated when abnormal traffic is routed through the network and notice logs are generated when anomalous signals, such as those from cyber-attacks, are generated. Regarding weird logs, split routing occurred 58 million and occupied approximately 33% of the total.An average of approximately 500 events per day accounted for Notice logs and they occupied 93.5% of the total of Notice log and address scan attacks for 57,116 logs. These cyber-attacks were used to determine specific IP addresses in the target network. Regarding protocols, TCP was used most often, accounting for 79.2% of the total, followed by UDP at 18.6% and ICMP at 2.1%. Regarding ports, port 443 accounted for 68% of the total, followed by port 25, 587, 2191, and 23.In terms of attacks, most attempts were made from South Korea, followed by Europe and the US. Improvements/Applications: We implemented Bro-IDS, a system for real-time detection and analysis of cyber threats on large-networks using the ELK stack. The results indicate that the proposed model has scalable security and can guarantee throughput corresponding to the bandwidth in a high-speed network environment.