Fast top-k query answering

Abstract

Efficient retrieval of the most relevant (i.e. top-k) tuples is an important requirement in information management systems which access large amounts of data. In general answering a top-k query request means to retrieve the k objects which score best for an objective function. We propose some improvements to the best position algorithm (BPA-2) [2]. To the best of our knowledge BPA-2 is currently the fastest available top-k query answering approach based on the widely known and applied Threshold Algorithm (short TA) of Fagin et al. [5]. Our proposed improvements lead to significantly reduced time and memory consumption and better scalability compared to BPA-2: (1) we dynamically create value rather than object based index structures out of the query restrictions posed by the user, (2) we introduce look-ahead techniques to process those index structures. While BPA-2 processes all pre-calculated indexes in parallel we always examine the most promising indexing structure next. We prototypically implemented our fast top-k query answering (FTA) approach. Our experiments showed an improvement by one to two orders of magnitude over BPA-2. © 2011 Springer-Verlag Berlin Heidelberg.