An OpenMP extension library for memory affinity

Abstract

OpenMP 4.0 extended affinity support to allow pinning of threads to places. Places are an abstraction of machine locations which in many cases do not require extensive hardware knowledge by the user. For memory affinity, i.e. data initialization and migration on NUMA systems, support is still missing in OpenMP. In this work we present an extension library for OpenMP which implements round-robin memory initialization over places and memory migration, either explicitly or implicitly. The latter is presented with an implementation based on a next-touch algorithm for Linux. We study the overhead of our methods with a simple model that allows to predict if migration is beneficial or not for a certain use case and we demonstrate the correctness of the migration methods and the correctness of our model prediction with the STREAM benchmark and an implementation of a CG method. Finally, we discuss how memory affinity could be integrated in future OpenMP versions.