Retrieval based on combining language models with clustering

Abstract

We propose a new retrieval method based on combining language models with clustering. The basic idea of the method is as follows. Firstly, documents in the collection are grouped into clusters by using a clustering algorithm. Secondly, clusters are imported into building language models which are used to estimate how likely a query could be generated from them. Thirdly, language models are smoothed by using a two-stage smoothing method. Our experiments show that the method outperforms both approach "purely" based on clustering and technique "purely" based on language model. © Spinger-Verlag 2004.