Research Cloud Bazaar: A software defined cloud workflow cost management tool

Abstract

Research workflows will benefit from a hybrid computing environment that offers seamless integration between on-campus and off-campus cloud resources. Commercial and Federal and commercial clouds provide researchers a rich set of computing platforms that allow opportunities to improve workflows and reduce the time to research the large number of cloud offerings, however, makes cost management, and workflow transitions to appropriate platforms challenging. Successfully mapping workflows from on-campus resources to the cloud and leveraging the available cost structures to find economical cost models are critical steps to enabling researcher access to this vast resource. To address these concerns, here we introduce the Research Computing Bazaar (RCB) software application for resource mapping and cost estimation. RCB is a software-as-a-service platform that is an elastic, scalable, and fault tolerant system. It is developed using actual data from research computing workloads and can be easily configured to be used by users or system administrators in computing environments that use Slurm. In this pilot, we inform researchers about opportunities offered by RCB to leverage flexible workload orchestration in managing cloud costs on a major cloud service provider. An extension into predictive capacities with machine learning mechanisms is being developed.