On fuzzy cluster validity indexes for high dimensional feature space

Abstract

Fuzzy document clustering aims at automatically organizing related documents into clusters in a flexible way. At this context, the topics identification addressed by documents in every cluster is performed by automatically discovering cluster descriptors, which are relevant terms present in these documents. Since documents are represented by a high-dimensional feature space, the extraction of good descriptors is a big problem to be solved. This problem is even bigger using fuzzy clustering, since the same descriptor can be representative for more than one cluster. Moreover, it is well-known that the Fuzzy C-Means clustering algorithm is also affected by documents dimensionality and the choice of correct partition of a given document collection into clusters is still a challenging problem. In order to overcome this drawback, we have investigated the most common fuzzy clustering validity indexes to validate the organization of data with high dimensional feature space, since they are commonly used to evaluate fuzzy clusters from low dimensional data sets.