Interpolation-based outlier detection for sparse, high dimensional data

Abstract

The clustering-based approach has always been an important research direction of outlier detection technology. The current detection approaches based on clustering of outliers are able to overcome the shortcoming of traditional test approach of outliers, however, most of the existing approach based on clustering to improve the choice of initial clustering center, this does not solve poor due to the sparse data clustering effect, and so cannot be radically improve the accuracy of detecting outliers. The data in the outlier detection problem can be regarded as a high degree of mixing between the normal point and the abnormal point. On the premise of reducing the loss of normal points, the set of outliers is the N samples farthest from the clustering center containing the most outliers. Inspired by this idea, an outlier detection approach based on genetic clustering for high-dimensional sparse data was proposed. By applying genetic algorithm based on traditional k-means, this approach processed the difference of original data and solved the problem of poor clustering effect caused by high-dimensional data sparsity. The experimental results show that compared with several improved k-means-based clustering approaches and outlier detection approaches, the proposed approach can reduce the loss of normal points and distinguish normal data and outlier more accurately.