A Novel Ensemble Method for Detecting Outliers in Categorical Data

Abstract

© 2020, World Academy of Research in Science and Engineering. All rights reserved. Outliers are data objects having very important and valuable information, but are rare in their datasets. Several algorithms are developed by various researchers for finding outliers from different types of datasets like multivariate datasets, time series datasets, image datasets and high dimensional datasets. These algorithms are specific to the type of datasets and no general purpose algorithm for detecting outliers in different types of datasets exists in the literature. Moreover most of the algorithms in the literature are capable of dealing with numerical data only. Real world datasets may contain data objects with categorical features in addition to numerical data objects. Here, we propose a novel ensemble learning method for finding outliers in categorical datasets, that ensemble one hot encoder and label encoder together with different outlier detection algorithms such as Local Outlier Factor, One ClassSupport Vector Machine, Elliptic Envelope, Isolation Forest and k-Nearest Neighbor. Experimental results using real world datasets show that the proposed ensemble method for finding outliers in categorical datasets outperforms the other outlier detection techniques.