Experimental Comparison of Memory Management Policies for NUMA Multiprocessors

Abstract

Nonuniformity of memory accessis an almost inevitable feature of the memory architecture in shared memory multiprocessor designs that can scale to large numbers of processors. One implication of NUMA architectures is that the placement and movement of code and data are crucial to performance. As memory architectures become more complex and the nonuniformity becomes less well hidden, system software must assume a larger role in providing memory management support for the programmer. This paper investigates the role of the operating system. We take an experimental approach to evaluating a wide-range of memory management policies. The target NUMA environment is BBN�s GP-1OOOmultiprocessor. Extensive local modifications have been made to the memory management subsystem of BBN�s nX operating system to support multiple policy implementations. Policy comparisons are based on the measured performance of real parallel applications. Our results show that there are memory management policies implemented in our system that can improve the performance of programs written using the simpler uniform memory access (UMA) programming model. While achieving the level of performance of a highly tuned NUMA program is still a difficult problem, some examples come close. There appears to be no single policy that can be considered the best over our set of test applications. Investigations into the contributions made by individual policy features toward overall behavior of the workload provide some insight into the design of a set of effective policies. © 1991, ACM. All rights reserved.