Anomaly detection in categorical datasets using bayesian networks

Abstract

In this paper we present a method for finding anomalous records in categorical or mixed datasets in an unsupervised fashion. Since the data in many problems consist of normal records with a small minority of anomalies, many approaches build a model from the training data and compare the test records against it. But instead of building a model, we keep track of the number of occurrences of different attribute value combinations. We also consider a more meaningful definition of anomalies and incorporate the Bayesian network structure in it. A scoring technique is defined for each test record. In this procedure we combine supports of different rules according to the Bayesian network structure in order to determine the label of the test instances. As it is shown in the results, our proposed method has a higher or similar f-measure and precision compared to a Bayesian network based approach in all cases. © 2011 Springer-Verlag.