A communication backend for parallel language compilers

Abstract

Generating good communication code is an important issue for all compilers targeting parallel or distributed systems. However, different compilers for the same parallel system usually implement the communication generation routines (e.g., message buffer packing) independently and from scratch. As a resuit, these compilers either pursue a simple approach (calling a standard runtime library), which does not do justice to the capabilities of the system, or they incur high development costs. This paper describes a way to separate the communication issues from other compilation aspects (e.g., determining the distribution of data and computation). This organization places the responsibility for communication issues with the communication backend, and this backend can be shared by different compilers. It produces code that is customized for each communication step, based on the exact data distribution and the characteristics of the target parallel system. This approach has several advantages: (1) The communication backend can be shared by multiple compilers, e.g., for different parallel languages. (2) The communication backend provides a way to integrate regular and irregular communication, e.g., as required to deal with irregular codes. (3) Retargeting of a parallel compiler is simplified, since the communication backend deats with the interface to communication (and the single-node compiler). (4) The communication backend can optimize the code, e.g., by constant folding and constant propagation. Code produced by the communication backend is always at least as fast as library code, but the customization has the potential to significantly improve performance depending on what information is known at compile time.