A twin-buffer scheme for high-throughput logging

Abstract

For a transactional database system, the efficiency of logging is usually crucial to its performance. The emergence of new hardware, such as NVM and SSD, eliminated the traditional I/O bottleneck of logging and released the potential of multi-core CPUs. As a result, the parallelism of logging becomes important. We propose a parallel logging subsystem called TwinBuf and implemented it in PostgreSQL. This solution can make better use of multi-core CPUs, and is generally applicable to all kinds of storage devices, such as hard disk, SSD and NVM. TwinBuf adopts per-thread logging slots to parallelize logging, and a twin-log-buffer mechanism to make sure that logging can be performed in a non-stop manner. It performs group commit to minimize the persistence overheads. Experimental evaluation was conducted to demonstrate its advantages.