täko: A Polymorphic Cache Hierarchy for General-Purpose Optimization of Data Movement

Abstract

Current systems hide data movement from software behind the load-store interface. Software's inability to observe and respond to data movement is the root cause of many inefciencies, including the growing fraction of execution time and energy devoted to data movement itself. Recent specialized memory-hierarchy designs prove that large data-movement savings are possible. However, these designs require custom hardware, raising a large barrier to their practical adoption. This paper argues that the hardware-software interface is the problem, and custom hardware is often unnecessary with an expanded interface. The täko architecture lets software observe data movement and interpose when desired. Specifcally, caches in täko can trigger software callbacks in response to misses, evictions, and writebacks. Callbacks run on reconfgurable dataflow engines placed near caches. Five case studies show that this interface covers a wide range of data-movement features and optimizations. Microarchitecturally, täko is similar to recent near-data computing designs, adding 5% area to a baseline multicore. täko improves per-formance by 1.4×-4.2×, similar to prior custom hardware designs, and comes within 1.8% of an idealized implementation.