Hierarchically clustered LSH for hierarchical outliers detection

Abstract

In this work we introduce hierarchical outliers that extend the notion of distance-based outliers for handling hierarchical data domains. We present a novel framework that permits us to detect hierarchical outliers in a consistent manner, providing a desired monotonicity property, which implies that a data observation that finds enough support so as to be disregarded as an outlier at a level of the hierarchy, will not be labelled as an outlier when examined at a more coarse-grained level above. This way, we enable users to grade how suspicious a data observation is, depending on the number of hierarchical levels for which the observation is found to be an outlier. Our technique utilizes an innovative locality sensitive hashing indexing scheme, where data points sharing the same hash value are being clustered. The computed centroids are maintained by our framework’s scheme index while detailed data descriptors are discarded. This results in reduced storage space needs, execution time and number of distance evaluations compared to utilizing a straightforward LSH index.