Halo: A Hybrid PMem-DRAM Persistent Hash Index with Fast Recovery

Abstract

Hash index, a fundamental component in many data management systems, can benefit from the emerging persistent memory (PMem) to achieve high performance and instant recovery. However, existing persistent hash indexes are suboptimal in at least three aspects. First, their performance suffers from the mismatch between small random write and access granularity of PMem hardware. Second, none of them are aware of the significance of write amplification caused by memory allocators and synchronization primitives. Third, hybrid designs (PMem+DRAM) focus on improving throughput at the cost of extremely long recovery time. In this paper, we present the design and implementation of Halo, a hybrid hash index for PMem+DRAM environment, featuring a specifically designed volatile index and log-structured persistent storage layout. In order to suppress write amplification caused by memory allocators and to facilitate recovery, we propose Halloc, a highly-efficient memory manager for Halo. In addition, we propose mechanisms such as batched writes, prefetching for hybrid reads, and reactive snapshot to further optimize performance. We conduct extensive evaluations on a 32-core platform equipped with Intel Optane DC Persistent Memory Modules. The results show that Halo achieves up to 17.5x and 81.2x higher read and write throughput than state-of-the-art hash indexes under a wide range of workloads. Halo also outperforms current hybrid designs in recovery speed, which is 1 to 2 orders of magnitude faster.