Pyramid: Revisiting memory extension with remote accessible non-volatile main memory

Abstract

Remote Direct Memory Access (RDMA) provides the ability to direct access remote user space memory without remote CPU’s involvement, shortening the network latency tremendously; in addition, a new generation of fast Non-Volatile Memory (NVM) technologies, such as 3D XPoint, is in production, and its property has the promise to access-speed like memory and durability-like storage. So, Remote access Non-Volatile Main Memory is reasonable. Traditional local memory extension is bounded by slow storage media (HDD/SSD). In this paper, first, we revisit local memory extension and propose a new memory extension model, Pyramid, extending memory with remote NVM; then, discussing the mechanism of remote data consistency, which can be delivered with RDMA operation of write-with-immediate in Pyramid; besides, we evaluate the performance of random access to remote NVM and manifest the performance opportunity brought by remote accessible NVM through comparing it with new technologies of storage-NVMe-SSD and PCM-based SSD. Finally, we argue that Pyramid promises memory scalability with good performance guarantee.