We propose the design and an implementation of a bulk parallel external memory priority queue to take advantage of both shared-memory parallelism and high external memory transfer speeds to parallel disks. To achieve higher performance by decoupling item insertions and extractions, we offer two parallelization interfaces: one using “bulk” sequences, the other by defining “limit” items. In the design, we discuss how to parallelize insertions using multiple heaps, and how to calculate a dynamic prediction sequence to prefetch blocks and apply parallel multiway merge for extraction. Our experimental results show that in the selected benchmarks the priority queue reaches 64% of the full parallel I/O bandwidth of SSDs and 49% of rotational disks, or the speed of sorting in external memory when bounded by computation.
CITATION STYLE
Bingmann, T., Keh, T., & Sanders, P. (2015). A bulk-parallel priority queue in external memory with STXXL. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 9125, pp. 28–40). Springer Verlag. https://doi.org/10.1007/978-3-319-20086-6_3
Mendeley helps you to discover research relevant for your work.