Cache replacement with dynamic exclusion

Abstract

A technique for reducing direct-mapped cache misses caused by conflicts for a particular cache line is introduced. A small finite state machine recognizes the common instruction reference patterns for which storing an instruction in the cache actually harms performance. Such instructions are dynamically excluded, that is, they are passed directly through the cache without being stored. This reduces misses to the instructions that would have been replaced. The effectiveness of dynamic exclusion is dependent on the severity of cache conflicts and thus on the particular program and cache size of interest. However, across the SPEC benchmarks, simulation results show an average reduction in miss rate of 33% for a 32-KB instruction cache with 16 B lines. Applying dynamic exclusion to one level of a cache hierarchy can improve the performance of the next level, since instructions do not need to be stored on both levels. Dynamic exclusion also improves combined instruction and data cache miss rates.