Hierarchical Models and Software Tools for Parallel Programming

Abstract

Hierarchically structured architectures are becoming more and more pervasive in the field of parallel and high performance computing. While memory hierarchies have been recognized for a long-time, only in the last years hierarchical parallel structures have gained importance, mainly as a result of the trend towards cluster architectures and high-performance application of computational grids. The similarity among the issues of managing memory hierarchies and those of parallel computation has been pointed out before (see for instance [213]). It is an open question if a single, unified model of both aspects exists , and if it is theoretically tractable. Correspondingly, a programming environment which includes support for both hierarchies is still lacking. We thus need well-founded models and efficient new tools for hierarchical parallel machines, in order to connect algorithm design and complexity results to high-performance program implementation. In this chapter we survey theoretically relevant results, and we compare them with existing software tools and programming models. One aim of the survey is to show that there are promising results with respect to the theoretical computational models, developed by merging the concepts of bulk-parallel computational models with those from the hierarchical memory field. A second goal is to investigate if software support has been realized, and what is still missing, in order to exploit the full performance of modern high-performance cluster architectures. Even in this case, solutions emerge from combining results of different nature, those employing hardware-provided shared memory and those explicitly dealing with message passing. We will see that both at the theoretical level and on the application side, combination of techniques from different fields is often promising, but still leaves many open questions and unresolved issues. The chapter is organized in three parts. The first one (Sect. 15.2) describes the architectural background of current parallel platforms and supercomput-ers. The basic architectural options of parallel architectures are explained, showing that they naturally lead to a hierarchy concept associated with the exploitation of parallelism. We discuss the technological reasons for, and future expectations of current architectural trends. The second part of the chapter gives an overview of parallel computational models, exploring the connection among the so-called parallel bridging models