PP-index: Using permutation prefixes for efficient and scalable approximate similarity search

Abstract

We present the Permutation Prefix Index (PP-Index), an in- dex data structure that allows to perform eficient approxi- mate similarity search. The PP-Index belongs to the family of the permutation- based indexes, which are based on representing any indexed object with \its view of the surrounding world", i.e., a list of the elements of a set of reference objects sorted by their distance order with respect to the indexed object. In its basic formulation, the PP-Index is strongly biased toward eficiency, treating effectiveness as a secondary as- pect. We show how the effectiveness can easily reach opti- mal levels just by adopting two \boosting" strategies: multi- ple index search and multiple query search. Such strategies have nice parallelization properties that allow to distribute the search process in order to keep high eficiency levels. We study both the eficiency and the effectiveness proper- ties of the PP-Index. We report experiments on collections of sizes up to one hundred million images, represented in a very high-dimensional similarity space based on the combi- nation of five MPEG-7 visual descriptors.