Ascending and descending order of random projections: Comparative analysis of high-dimensional data clustering

Abstract

Random Projection has been used in many applications for dimensionality reduction. In this paper, a variant to the iterative random projection K-means algorithm to cluster high-dimensional data has been proposed and validated experimentally. Iterative random projection K-means (IRP K-means) method [1] is a fusion of dimensionality reduction (random projection) and clustering (K-means). This method starts with a chosen low-dimension and gradually increases the dimensionality in each K-means iteration. K-means is applied in each iteration on the projected data. The proposed variant, in contrast to the IRP K-means, starts with the high dimension and gradually reduces the dimensionality. Performance of the proposed algorithm is tested on five high-dimensional data sets. Of these, two are image and three are gene expression data sets. Comparative Analysis is carried out for the cases of K-means clustering using RP-Kmeans and IRP-Kmeans. The analysis is based on K-means objective function, that is the mean squared error (MSE). It indicates that our variant of IRP K-means method is giving good clustering performance compared to the previous two (RP and IRP) methods. Specifically, for the AT & T Faces data set, our method achieved the best average result (9.2759× 109), where as IRP-Kmeans average MSE is 1.9134× 1010. For the Yale Image data set, our method is giving MSE 1.6363× 108, where as the MSE of IRP-Kmeans is 3.45× 108. For the GCM and Lung data sets we have got a performance improvement, which is a multiple of 10 on the average MSE. For the Luekemia data set, the average MSE is 3.6702× 1012 and 7.467× 1012 for the proposed and IRP-Kmeans methods respectively. In summary, our proposed algorithm is performing better than the other two methods on the given five data sets.