Performance comparison of language models for information retrieval

Abstract

Vector Space Model (VSM), Statistical Language Model (SLM) and Inference Network are three distinguished language models. Instead of evaluating their performance directly, we estimate the information strategies founded on them using the known measures: precision and recall. What's more, we proposed the Sort Order Rationality (SOR) to make further performance comparison among different language models. All models are tested on a standard testing collection. Three important conclusions are attained: (1). The IR model combining the statistical language modeling and inference network approaches is better than that only founded on statistical language modeling approach. What's more, it is also better than that based on vector space modeling approach. (2). The performance of IR model based on VSM is similar to that based on SLM. (3). The Dirichlet priors method often is a better option to smooth a statistical language model. In some respects, these conclusions provide some experimental bases for constructing an efficient information retrieval system. © 2005 by International Federation for Information Processing.