Text Retrieval Quality : A Primer

Abstract

http://www.oracle.com/technetwork/database/enterprise-edition/imt-quality-092464.html Text retrieval engines, popularly known as search engines, return a list of documents (the hitlist) for a query. Typically there are some good documents in the list and some bad ones. The quality of a search engine is measured in terms of the proportion of good hits in the list, the positions of good hits relative to bad ones, and the proportion of good documents missing from the list. Ideally, a search engine must return all the good documents and only the good documents. Such an engine has very good quality and is said to have high precision, recall, and utility. Real search engines are only able to return some of the good documents in the collection along with some bad ones. This paper explains quality metrics such as precision and recall. It also describes the TREC quality benchmark and explains how to interpret TREC results. Good and Bad: Correctness vs. Relevance Every structured query exactly defines a set of rows to be retrieved. Every row retrieved by the database for a structured query is always correct. There is never a bad hit for a structured query.