transAD: An anomaly detection network intrusion sensor for the web

Abstract

Content-based Anomaly Detection (AD) techniques are regarded as a promising mechanism to detect ‘zero-day’ attacks. AD sensors have also been shown to perform better than signature-based systems in detecting polymorphic attacks. However, the False Positive Rates (FPRs) produced by current AD sensors have been a cause of concern. In this paper, we introduce and evaluate transAD, a system of network traffic inspection AD sensors that are based on Transductive Confidence Machines (TCM). Existing TCM-based implementations have very high FPRs when used as NIDS. Our approach leverages an unsupervised machine-learning algorithm to identify anomalous packets and thus, unlike most AD sensors, transAD does not require manually labeled data. Moreover, transAD uses an ensemble of TCM sensors to achieve better detection rates and lower FPRs than single sensor implementations. Therefore, transAD presents a hardened defense against poisoning attacks. We evaluated our prototype implementation using two real-world data sets collected from a public university’s network. TransAD processed approximately 1.1 million packets containing real attacks. To compute the ground truth, we manually labeled 18,500 alerts. In the course of scanning millions of packets, our sensor’s low FPR would significantly reduce the number of false alerts that need to be inspected by an operator, while maintaining a high detection rate.