Large-scale traffic anomaly detection: Analysis of real netflow datasets

Abstract

The analysis of large amount of traffic data is the daily routine of Autonomous Systems and ISP operators. The detection of anomalies like denial-of-service (DoS) or distributed denial-of-service (DDoS) is also one of the main issues for critical services and infrastructures. The suitability of metrics coming from the information theory for detecting DoS and DDoS episodes has been widely analyzed in the past. Unfortunately, their effectiveness are often evaluated on synthetic data set, or, in other cases, on old and unrepresentative data set, e.g. the DARPA network dump. This paper presents the evaluation by means of main metrics proposed in the literature of a real and large network flow dataset, collected from an Italian transit tier II Autonomous System (AS) located in Rome. We show how we effectively detected and analyzed several attacks against Italian critical IT services, some of them also publicly announced. We further report the study of others legitimate and malicious activities we found by ex-post analysis.