Regional out-of-order writes in total store order

Abstract

The store buffer, an essential component in today's processors, isdesigned to hide memory latency by moving stores off the processor's critical path. Furthermore, under the Total Store Order (TSO)memory model, the store buffer ensures the in-order retirement ofstores. Problems arise when the store buffer is full or, under TSO,when the leading store encounters a cache miss, which blocks allsubsequent stores and incurs severe performance bottlenecks.This work presents a software-hardware co-designed approachto cope with this bottleneck for processors with strong consistencyguarantees. Our proposal is driven by the insight that store operations can be reordered if their reordering does not change theobservable program behavior. The compiler delineates safe regionswithin which stores can be shuffled while still delivering the sameobservable behavior as if they performed in program order andunsafe regions within which stores must be kept in program order.This is leveraged by a novel dual-mode store buffer that switchesbetween the out-of-order and in-order execution of stores withinthe safe and respectively unsafe regions. Correctness is preservedthrough well-placed fences inserted by the compiler, which impedethe execution of stores from the following regions until all storesof the current region complete. Our dual-mode store buffer onlyrequires one extra bit per entry, significantly decreases processorstall cycles, and brings 8.13% performance improvements comparedto a mainstream store buffer.