Leveraging caches to accelerate hash tables and memoization

Abstract

Hash tables are widely used, but they are inefficient in current systems: they use core resources poorly and suffer from limited spatial locality in caches. To address these issues we propose HTA, a technique that accelerates hash table operations via simple ISA extensions and hardware changes. HTA adopts an efficient hash table format that leverages the characteristics of caches. HTA accelerates most operations in hardware, and leaves rare cases to software. We present two implementations of HTA, Flat-HTA and Hierarchical- HTA. Flat-HTA adopts a simple, hierarchy-oblivious layout and reduces runtime overheads with simple changes to cores. Hierarchical-HTA is a more complex implementation that uses a hierarchy-aware layout to improve spatial locality at intermediate cache levels. It requires some changes to caches and provides modest benefits over Flat-HTA. We evaluate HTA on hash table-intensive benchmarks and use it to accelerate memoization, a technique that caches and reuses the outputs of repetitive computations. Flat-HTA improves the performance of the state-of-the-art hash table-intensive applications by up to 2, while Hierarchical-HTA outperforms Flat-HTA by up to 35%. Flat-HTA also outperforms software memoization by 2.