Faster MaxScore document retrieval with aggressive processing

Abstract

Large search engines process thousands of queries per second over billions of documents, making a huge performance gap between disjunctive and conjunctive queries. An important class of optimization techniques called top-k processing is therefore used to narrow the gap. In this paper, we propose an aggressive algorithm based on the document-at-a-time (DAAT) MaxScore, aiming at further reducing the query latency of disjunctive queries. Essentially, our approach, named Aggressive MaxScore (AMaxScore), can speed up quickly by fine-tuning the initial top-k threshold, which allows a first aggressive process and then a supplementary process if not enough results are returned. Experiments with TREC GOV2 collection show that our approach reduces disjunctive query processing time by almost 15.4% on average over the state-of-the-art MaxScore baseline, while still returns the same results as the disjunctive evaluation. © 2014 Springer International Publishing Switzerland.