A fast PQ hash code indexing

Abstract

This paper presents a Compressed PQ Indexing (CPQI) data structure, which realizes the further compression of sparse entries, requires only sub-linear search time, and the sparse entries are no longer stored. The proposed CPQI saves storage space and is suitable for in-memory computing for large-scale data. The CPQI employs the Minimal Perfect Hash to hash the PQ code, preserve non-null entries, and store the structure very compactly; the compressed PQ hash code index no longer stores PQ code. A sub-linear time search is implemented by combining Bloom filtering with a minimum perfect hash function.