Using hit curves to compare search algorithm performance

2Citations
Citations of this article
13Readers
Mendeley users who have this article in their library.

Abstract

Databases continue to grow but the metrics available to evaluate information retrieval systems have not changed. Large collections such as MEDLINE and the World Wide Web contain many relevant documents for common queries. Ranking is therefore increasingly important and successful information retrieval systems, such as Google, have emphasized ranking. However, existing evaluation metrics such as precision and recall, do not directly account for ranking. This paper describes a novel way of measuring information retrieval performance using weighted hit curves adapted from the field of statistical detection to reflect multiple desirable characteristics such as relevance, importance, and methodologic quality. In statistical detection, hit curves have been proposed to represent occurrence of interesting events during a detection process. Similarly, hit curves can be used to study the position of relevant documents within large result sets. We describe hit curves in light of a formal model of information retrieval, show how hit curves represent system performance including ranking, and define ways to statistically compare performance of multiple systems using hit curves. We provide example scenarios where traditional measures are less suitable than hit curves and conclude that hit curves may be useful for evaluating retrieval from large collections where ranking performance is crucial. © 2006 Elsevier Inc. All rights reserved.

Cite

CITATION STYLE

APA

Herskovic, J. R., Iyengar, M. S., & Bernstam, E. V. (2007). Using hit curves to compare search algorithm performance. Journal of Biomedical Informatics, 40(2), 93–99. https://doi.org/10.1016/j.jbi.2005.12.007

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free