Characterization and understanding machine-specific interconnects

Abstract

The use of efficient communication patterns is becoming increasingly important to utilize high concurrency and achieve high throughput and low latency of modern high-performance supercomputers. Efficacy is not only dictated by an application communication pattern, but also driven by the interconnect properties, the node architecture, and the quality of runtime communication libraries. Different systems require different tradeoffs with respect to communication mechanisms, which can impact the choice of application implementations. We use the in-house MPI benchmark suite to study the communication behavior of the interconnects and guide the performance tuning of scientific applications. We report the results of our investigation of four supercomputer systems located at ALCF, and present lessons learned from our experience. © 2013 Springer-Verlag Berlin Heidelberg.