Informativeness measures have been used in interactive information retrieval and automatic summarization evaluation. Indeed, as opposed to adhoc retrieval, these two tasks cannot rely on the Cranfield evaluation paradigm in which retrieved documents are compared to static query relevance document lists. In this paper,we explore the use of informativeness measures to evaluate adhoc task. The advantage of the proposed evaluation framework is that it does not rely on an exhaustive reference and can be used in a changing environment in which new documents occur, and for which relevance has not been assessed. We show that the correlation between the official system ranking and the informativeness measure is specifically high for most of the TREC adhoc tracks.
CITATION STYLE
Deveaud, R., Moriceau, V., Mothe, J., & Juan, E. S. (2016). Informativeness for adhoc IR evaluation: A measure that prevents assessing individual documents. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 9626, pp. 818–823). Springer Verlag. https://doi.org/10.1007/978-3-319-30671-1_73
Mendeley helps you to discover research relevant for your work.