Improved Vectorization for Efficient Chemistry Computations in OpenFOAM for Large Scale Combustion Simulations

Abstract

The computation of chemical reaction rates is commonly the performance bottleneck in CFD simulations of turbulent combustion with detailed chemistry. Therefore, an optimization method is used where C++ source code is automatically generated for arbitrary reaction mechanisms. The generated code is highly optimized for the chosen mechanism and contains all routines for computing chemical reaction rates. In this work, the serial performance of the automatically generated source code, which in an earlier work only used ISO C++, is further improved by utilizing two compiler extensions: restrict and __builtin_assume_aligned. Introducing these two extensions to the generated code reduces the time for computing chemical reaction rates by up to 50% for the investigated reaction mechanism and total simulation time by up to 25%. Compared to OpenFOAM's standard chemistry implementation, the new code is faster by a factor of 10. This work discusses the effect of the two compiler extensions on performance by looking at two specific kernel functions from the automatically generated code and the effect on the assembly generated by the gcc and Intel compilers. The newly optimized code is used to evaluate the performance gain in a large scale parallel case, which simulates an experimentally investigated turbulent flame of laboratory scale on 14,400 CPU cores on the Hazel Hen cluster at HLRS. In the simulation, no combustion models are used and the flame is resolved down to the smallest length scales. With this approach, comparison of measured data with the simulation shows very good agreement. Using the optimized code including compiler extensions, total simulation time decreases by 20% compared to the same code without compiler extensions. A comprehensive database from the simulation results has been assembled and will consist of 10 TB of 3D and 2D transient field variables.