TADOOP: Mining network traffic anomalies with Hadoop

Abstract

Today, various anomalies and large number of flows in a network make traffic anomaly detection a big challenge. In this paper, we propose DTE-FP (Dual q Tsallis Entropy for flow Feature with Properties), a more efficient method for traffic anomaly detection. To handle huge amount of traffic, based on Hadoop, we implement a network traffic anomaly detection system named TADOOP, which supports semiautomatic training and both offline and online traffic anomaly detection. TADOOP with a cluster of five servers has been deployed in Tsinghua University Campus Network. Furthermore, we compare DTE-FP with Tsallis entropy, and the experimental results show that DTE-FP has much better detection capability than Tsallis entropy.