Low space external memory construction of the succinct permuted longest common prefix array

Abstract

The longest common prefix (LCP) array is a versatile auxiliary data structure in indexed string matching. It can be used to speed up searching using the suffix array (SA) and provides an implicit representation of the topology of an underlying suffix tree. The LCP array of a string of length n can be represented as an array of length n words, or, in the presence of the SA, as a bit vector of 2n bits plus asymptotically negligible support data structures. External memory construction algorithms for the LCP array have been proposed, but those proposed so far have a space requirement of O(n) words (i.e. O(n log n) bits) in external memory. This space requirement is in some practical cases prohibitively expensive. We present an external memory algorithm for constructing the 2n bit version of the LCP array which uses O(n log σ) bits of additional space in external memory when given a (compressed) BWT with alphabet size σ and a sampled inverse suffix array at sampling rate O(log n). This is often a significant space gain in practice where σ is usually much smaller than n or even constant. The algorithm has average run-time O(n log n log σ) and worst case run-time O(n2 log σ). It can be improved to O(n log2 n log σ) worst case time while keeping the same space bound in external memory if O(n/ log n) bits of internal memory are available. We also present experimental data showing that our approach is practical.