Retrieval effectiveness of cross language information retrieval search engines

Abstract

This study evaluates the retrieval effectiveness of English-Chinese (EC) cross-language information retrieval (CLIR) on four common search engines along the dimensions of recall and precision. We formulated a set of simple and complex queries on different topics including queries with translation ambiguity. Three independent bilingual proficient evaluators reviewed a total of 960 returned web pages each to assess document relevance. Findings showed that CLIR effectiveness is poor with average recall and precision values of 0.165 and 0.539 for monolingual EE/CC searches, and 0.078 and 0.282 for cross lingual CE/EC searches. Google outperformed Yahoo! in the experiments, and EC and EE searches returned better results than CE and CC results respectively. As this is the first set CLIR retrieval effectiveness measurements reported in literature, these findings can serve as a benchmark and provide a better understanding of the current CLIR capabilities of Web search engines. © 2011 Springer-Verlag.