Short Text Mapping Based on Fast Clustering Using Minimum Spanning Trees

Abstract

Due to short length and limited content, short text representation has the problem of high-dimension and high-sparsity. For the purpose of achieving the goal of reducing the dimension and eliminate the sparseness while preserve the semantics of the information in the text to be represented, a method of short text mapping based on fast clustering using minimum spanning trees is proposed. First, we remove the irrelevant terms, then a clustering method based on minimum spanning tree is adopted to identify the relevant term set and remove the redundant terms to get the short text mapping space. Finally, a matrix mapping method is designed to represent the original short text on a highly correlated and non-redundant short text mapping space. The proposed method not only has low time complexity but also produces higher quality short text mapping space. The experiments prove that our method is feasible and effective.