Clusterability and centroid approximation

Abstract

Identifying clusters in a dataset is valuable. Most existing data clustering algorithms need the number of clusters as an input. The present paper introduces a graphical method that outputs the number of clusters. Once the number of clusters is calculated, other clustering algorithms may use it. This method is cubic in the number of input data. Given that most of the databases are extremely big, it is convenient to choose a random sample to make a faster detection of the clusters. However, not only is speed important but the level of confidence in this method is as well. So the level of accuracy is calculated for each run of the algorithm. Three different sampling algorithms have been used in order to choose the random sample. The efficiency of the three algorithms has been compared based on the number of clusters that they detect and the running time. In addition, depending on the number of points in each cluster, central points or spies that cover the clusters are assigned to each cluster. In order to have a complete vision of the evolution of data clustering and the future tendencies of the clusters, a time series analysis was conducted as well. All the tests have been conducted using datasets from the UCI Machine learning repository and artificially generated datasets. We present the experimental results and show the effect of sampling algorithms and the number of clusters.