Integration of compute-intensive tasks into scientific workflows in BeesyCluster

Abstract

The paper presents design, implementation details and simulations of scientific workflows involving compute-intensive tasks on clusters and PCs. The author has incorporated support for scientific workflows into previously developed J2EE-based BeesyCluster, deployed at Academic Computer Center Gdansk Poland on large HPC resources including a large 288-processor Itanium2 cluster. BeesyCluster allows users to manage various accounts on clusters/PCs via WWW/ Web Services, run shell interactively, compile, queue, run tasks, publish services for other users, work in teams. A frequent scenario in HPC computing is analyzed, in which a workflow is combined from tasks offered by different users. Steps of the workflow include data preparation and following simulations run in parallel on clusters, with and without queuing systems. © Springer-Verlag Berlin Heidelberg 2006.