Improving direct-mapped cache performance by the addition of a small fully-associative cache prefetch buffers

Abstract

Projections of computer technology forecast processors with peak performance of 1,000 MIPS in the relatively near future. These processors could easily lose half or more of their performance in the memory hierarchy if the hierarchy design is based on conventional caching techniques. This paper presents hardware techniques to improve the performance of caches. Miss caching places a small fully-associative cache between a cache and its refill uath. Misses in the cache that hit in the miss cache ha;e only a one cycle miss penalty, as o ii posed to a many cycle miss penalty without the mns cat e. Small miss caches of 2 to 5 entries are shown to be very effective in removing mapping conflict misses in first-level direct-mapped caches. is an improvement to miss caching -associative cache with the victim of a miss and not t e requested line. Small victim caches of 1 to 5 entries are even more effective at removing conflict misses than miss caching. Stream buffers refetch cache lines statting at a cache miss address. fh e prefetched data is placed in the buffer and not in the cache. Stream buffers are useful in removing capacity and compulsory cache misses, as well as some instruction cache conflict misses. Stream buffers are more effective than previously investigated next slower level in the An extension to prefetching along multiple intertwined data reference sueaIns. Together, victim caches and stream buffers reduce the miss rate of the first level in the cache hierarchy by a factor of two to three on a set of six large benchmarks.