STXXL: Standard Template Library for XXL data sets

Abstract

We present a software library STXXL, that enables practice-oriented experimentation with huge data sets. STXXL is an implementation of the C++ standard template library STL for external memory computations. It supports parallel disks, overlapping between I/O and computation, and pipelining technique that can save more than half of the I/Os. STXXL has already been used for computing minimum spanning trees, connected components, breadth-first search decompositions, constructing suffix arrays, and computing social network analysis metrics. © Springer-Verlag Berlin Heidelberg 2005.