Rules Refine the Riddle: Global Explanation for Deep Learning-Based Anomaly Detection in Security Applications

Abstract

Deep learning (DL) based anomaly detection has shown great promise in the field of security due to its remarkable performance in various tasks. However, the issue of poor interpretability in DL models has significantly impeded their deployment in practical security applications. Despite the progress made in existing studies on DL explanations, the majority of them focus on providing local explanations for individual samples, neglecting the global understanding of the model knowledge. Furthermore, most explanations for supervised models fail to apply to anomaly detection due to their different learning mechanisms. In this work, we address the gap in the existing research by proposing GEAD, a novel global explanation for DL-based anomaly detection, to extract high-fidelity rules from DL models. We apply GEAD to two security applications, network intrusion detection and system log anomaly detection, and demonstrate the efficacy with three usages: comparing model knowledge with expert knowledge, identifying knowledge discrepancies between models, and combining model and expert knowledge. We provide several case studies to showcase how GEAD can significantly enhance existing anomaly detection systems. Moreover, we provide a real-world deployment in a SCADA system to showcase the potential in practice. Some important insights are drawn to help the community understand and improve anomaly detection systems in security.