Distance Measures for Clustering of Documents in a Topic Space

Abstract

Topic modeling is a method for discovery of topics (groups of words). In this paper we focus on clustering documents in a topic space obtained using the MALLET tool. We tested several different distance measures with two clustering algorithm (spectral clustering, agglomerative hierarchical clustering) and described those that served better (cosine distance, correlation distance, bhattacharyya distance) than the Euclidean metric for k-means algorithm. For evaluation purpose we used Adjusted Mutual Information (AMI) score. The need for such experiments comes from the difficulty of choosing appropriate grouping methods for the given data, which is specific in our case.