CDA: Succinct spaghetti

Abstract

A pivot table is a popular mechanism for building indexes for similarity queries. Precomputed distances to a set of references are used to filter non-relevant candidates. Every pivot serves as a reference for all, or a proper subset of, the objects in the database.Each pivot filters its share of the database and the candidate list for a query is the intersection of all the partial lists.The spaghetti data structure is a mechanism to compute the above intersection without performing a sequential scan over the database, and consist of a collection of circular linked lists. In this paper, we present a succinct version of the spaghetti. The proposed data structure uses less memory and, unlike the original spaghetti, it can compute the intersection using an arbitrary order of the component sets. This later property enables more sophisticated evaluation heuristics leading to faster intersection computation. We present the analysis of the performance, as well as a comprehensive set of experiments where the new approach is proven to be faster in practice.