Effective clustering algorithm for high-dimensional sparse data based on SOM

Abstract

With increasing opportunities for analyzing large data sources, we have noticed a lack of effective processing in datamining tasks working with large sparse datasets of high dimensions. This work focuses on this issue and on effective clustering using models of artificial intelligence. The authors of this article propose an effective clustering algorithm to exploit the features of neural networks, and especially Self Organizing Maps (SOM), for the reduction of data dimensionality. The issue of computational complexity is resolved by using a parallelization of the standard SOM algorithm. The authors have focused on the acceleration of the presented algorithm using a version suitable for data collections with a certain level of sparsity. Effective acceleration is achieved by improving the winning neuron finding phase and the weight actualization phase. The output presented here demonstrates sufficient acceleration of the standard SOM algorithm while preserving the appropriate accuracy.