Accelerating xor-based erasure coding using program optimization techniques

Abstract

Erasure coding (EC) affords data redundancy for large-scale systems. XOR-based EC is an easy-To-implement method for optimizing EC. This paper addresses a significant performance gap between the state-of-Theart XOR-based EC approach (4.9 GB/s coding throughput) and Intels high-performance EC library based on another approach (6.7 GB/s). We propose a novel approach based on our observation that XOR-based EC virtually generates programs of a Domain Specific Language for XORing byte arrays. We formalize such programs as straight-line programs (SLPs) of compiler construction and optimize SLPs using various program optimization techniques. Our optimization flow is three-fold: 1) reducing the number of XORs using grammar compression algorithms; 2) reducing memory accesses using deforestation, a functional program optimization method; and 3) reducing cache misses using the (red-blue) pebble game of program analysis. We provide an experimental library, which outperforms Intels library with an 8.92 GB/s throughput.