Threat Alert Prioritization Using Isolation Forest and Stacked Auto Encoder with Day-Forward-Chaining Analysis

Abstract

Security Incident and Event Manager (SIEM) is a security management approach designed to identify possible threats within a real-time enterprise environment. The main challenge for SIEM is to find critical security incidents among a huge number of less critical alerts coming from separate security products. The continuously growing number of internet-connected devices has led to the alert fatigue problem, which is defined as the inability of security operators to investigate each incoming alert from intrusion detection systems. This fatigue can lead to human errors and leave many alerts being not investigated. Aiming at reducing the number of less important threat alerts presented to security operators, this paper presents a new method for highlighting critical alerts with a minimal number of false negatives. The proposed method employs isolation forest to ensure unsupervised performance and adaptability to different types of networks. Furthermore, it takes the advantage of day-forward-chaining analysis to ensure the detection of highly important alerts in real time. The number of false positive cases is reduced by employing an autoencoder. The proposed method achieved a recall score of 95.89% and a false positive rate of 5.86% on a dataset comprising more than half a million alerts collected in a real-world enterprise environment over ten months. This study highlights the importance of addressing the alert fatigue problem and validates the effectiveness of unsupervised learning in filtering out less important threat alerts.