Particle filtering as a modeling tool for anomaly detection in networks

Abstract

When linearity can be rigorously assumed for stochastic processes, the linear Kalman filter can be used as a powerful tool for anomaly detection in communication networks. However, this assumption done with a strong evidence is not generally proved in a rigorous way. So it is important to develop other methodology, for the scope of anomaly detection, which are not obliged to be based on that assumption. This paper is focused on the use of particle filtering to build a normal behavioral model for an anomaly detector. The particle filter is calibrated for entropy reduction for the scope of noise reduction in the measurements. With the help of a mixture of normal distributions, we can reuse the filtered observations to identify anomalous events in a few number of classes. Generally anomalies might be rare and thus they might happen on a few clusters. So, using a new decision process based on a hidden markov model, we can track and identify the potential abnormal clusters. We study the performances of this system by analyzing the false alarm rate vs detection rate trade-off by means of Receiver Operating Characteristic curve, and compare the results with the Kalman filter. We validate the approach to track volume anomalies over real network traffic.