A Clustering Algorithm for Large Datasets Based on Detection of Density Variations

Abstract

Clustering algorithms help handle unlabeled datasets. In large datasets, density-based clustering algorithms effectively capture the intricate structures and varied distributions that these datasets often exhibit. However, while these algorithms can adapt to large datasets by building clusters with arbitrary shapes by identifying low-density regions, they usually struggle to identify density variations. This paper proposes a Variable DEnsity Clustering Algorithm for Large datasets (VDECAL) to address this limitation. VDECAL introduces a large-dataset partitioning strategy that allows working with manageable subsets and prevents workload imbalance. Within each partition, relevant objects subsets characterized by attributes such as density, position, and overlap ratio are computed to identify both low-density regions and density variations, thereby facilitating the building of the clusters. Extensive experiments on diverse datasets show that VDECAL effectively detects density variations, improving clustering quality and runtime performance compared to state-of-the-art DBSCAN-based algorithms developed for clustering large datasets.