Parallel coupling of heterogeneous domains with KOP3D using PACX-MPI

Abstract

This paper outlines how coupled heterogeneous domains can be distributed in a supercomputing environment using PACX-MPI. KOP3D is a coupling program in the field of aero acoustics, a typical multi-scale application, since on the one hand it has to account for the small vortical structures as the source of the noise and on the other for the long wave length of the acoustic waves. The amount of energy, and with that the pressure, in the sound generating flow is by orders of magnitude higher than the amount of energy carried by the acoustic waves. The acoustical wavelength is much larger than the diameter of the noise-generating vortices, and even the computational domains may vary in a wide range from the geometry which is in the order of meters for an airfoil to the distance between a starting or landing plan and the observer on the ground which is in the order of about 1 km. Through the use of heterogeneous domain decomposition it is possible to reduce the computation time needed to simulate large flow fields by adaptation of the equations, the discretization, the mesh and the time step to the local requirements of the flow simulation within each sub-domain. These locally adapted computations may result in different requirements on the computer architecture. On the other side, in a supercomputing network there are generally different computer architectures available. By matching the sub-domains to the best suited available architectures an even shorter time to solution may be gained. The parallel version of the KOP3D coupling scheme is shown and the benefits of running the simulation distributed on the vector machine NEC-SX8 and on the scalar Itanium II (IA64) machine are demonstrated. A 2D and a 3D testcase are presented. © 2009 Springer-Verlag Berlin Heidelberg.