Exploring HPC-based scientific software as a service using CometCloud

Abstract

The use of in-silico simulations in experimental science can greatly increase laboratory efficiency and provide additional insights into interactions not easily described by traditional methods. Such simulations require significant amounts of computational resources, accessible only via supercomputers of large-scale high-performance clusters. Due to the complexity of the computational experiments, as well as the usage of the underlying resources, experimental scientists heavily rely on computational scientists with HPC expertise to perform these simulations. This additional bottleneck prevents the widespread adoption of real time in-silico simulation as a driver for laboratory experimentation. In this paper, we aim to overcome this bottleneck by presenting the architecture of an end-to-end framework to enable HPC Software as a Service. This framework is designed to make it easy for scientific applications to run on top of dynamically federated HPC resources. The framework enables HPC resource sharing while maximizing throughput and utilization. We focus specifically on a use case where an experimental scientist uses a mobile portal to control dissipative particle dynamics experiments that are executed on a remote supercomputer (IBM Blue Gene/Q).