EPOCH: Enabling Preemption Operation for Context Saving in Heterogeneous FPGA Systems

Abstract

FPGAs have become prevalent in multi-tenant cloud environments to offload compute-intensive tasks from the main CPU. The operating system (OS) plays a vital role in identifying tasks suitable for offloading and coordinating between the CPU and FPGA for seamless task execution. The OS leverages preemption to manage CPU efficiently and balance CPU time; however, preempting tasks running on FPGAs without context loss remains challenging. Despite growing reliance on FPGAs, vendors have yet to deliver a solution that fully preserves and restores task context. In this paper, we present EPOCH, a practical out-of-the-box framework designed to seamlessly preserve the state of tasks running on multi-tenant cloud FPGAs. EPOCH enables interrupting a tenant's execution at any arbitrary clock cycle, capturing its state, and saving this 'state snapshot' in off-chip memory with fine-grain granularity. Subsequently, when task resumption is required, EPOCH can resume execution from the saved 'state snapshot', eliminating the need to restart the task from scratch. EPOCH automates intricate processes, shields users from complexities, and synchronizes all underlying logic in a common clock domain, mitigating timing violations and ensuring seamless handling of interruptions. We thoroughly verified EPOCH using diverse benchmarks, demonstrating the significance of the proposed work in a real-world scenario with minimal area overhead. EPOCH proficiently captures the state of fundamental FPGA elements, such as look-up tables, flip-flops, block-RAMs, and digital signal processing units. On ZynQ-XC7Z020 SoC, the proposed solution achieves context save and restore operations per frame in 62.2 and 67.4 μs, respectively.