A fuzzy neighborhood rough set method for anomaly detection in large scale data

Abstract

Recent research studies on outlier detection have focused on examining the nearest neighbor structure of a data object to measure its outlierness degree. Moreover, popular outlier detection methods require the pairwise comparison of objects to compute the nearest neighbors. This quadratic problem is not scalable to large data sets, making multidimensional outlier detection for big data still an open challenge. In this article, we present a new approach for outlier detection, based on highly scalable approach to compute the nearest neighbors of objects using fuzzy rough set theory. At the same time, the outlier ranking process is accelerated by using a high-performance and a parallel computating using mapreduce framework.