Performance assessment and parallelisation issues of the CFD code NSMB

Abstract

We present results of an extensive comparative benchmarking study of the numerical simulation code NSMB for computational fluid dynamics (CFD), which is parallelised on the level of domain decomposition. The code has a semi-industrial background and has been ported to and optimised for a variety of different computer platforms, allowing us to investigate both massively-parallel microprocessor architectures (Cray XT3, IBM SP4) and vector machines (NEC SX-5, NEC SX-8). The studied test configuration represents a typical example of a three-dimensional time-resolved turbulent flow simulation. This is a commonly used test case in our research area, i. e., the development of methods and models for accurate and reliable time-resolved flow simulations at relatively moderate computational cost. We show how the technique of domain decomposition of a structured grid leads to an inhomogeneous load balancing already at a moderate CPU-to-block-count ratio. This results in severe performance limitations for parallel computations and inhibits the efficient usage of massively-parallel machines, which are becoming increasingly ubiquitous in the high-performance computing (HPC) arena. We suggest a practical method to alleviate the load-balancing problem and study its effect on the performance and related measures on one scalar (Cray XT3) and one vector computer (NEC SX-8). Finally, we compare the results obtained on the different computation platforms, particularly in view of the improved load balancing, computational efficiency, machine allocation and practicality in everyday operation. © Springer-Verlag Berlin Heidelberg 2010.