Alphabet partitioning for compressed rank/select and applications

Abstract

We present a data structure that stores a string s[1..n] over the alphabet [1..σ] in nH0(s) + o(n)(H0(s) + 1) bits, where H0(s) is the zero-order entropy of s. This data structure supports the queries access and rank in time O(lg lg σ), and the select query in constant time. This result improves on previously known data structures using o(n lg σ) bits, where on highly compressible instances the redundancy o(n lg σ) cease to be negligible compared to the nH0(s) bits that encode the data. The technique is based on combining previous results through an ingenious partitioning of the alphabet, and practical enough to be implementable. It applies not only to strings, but also to several other compact data structures. For example, we achieve (i) faster search times and lower redundancy for the smallest existing full-text self-index; (ii) compressed permutations π with times for π() and π-1() improved to log-logarithmic; and (iii) the first compressed representation of dynamic collections of disjoint sets. © 2010 Springer-Verlag.