The effect of the 2nd generation clusters: Changes in the parallel programming paradigms

Abstract

Programming paradigms for networks of symmetric multi-processor (SMP) workstation (2nd generation of clusters) are discussed and a new paradigm is introduced. The SMP cluster environments are explored in regard to their advantages and drawbacks with a special focus on memory architectures and communication. The new programming paradigm provides a solution to write efficient parallel applications for the 2nd generation of clusters. The paradigm aims at improving the overlap of computation and communication and the locality of communication operations. The preliminary results with large message sizes indicate improvements in excess of 30% over traditional MPI implementations. © Springer-Verlag 2004.