Continuous integration, delivery, and deployment for scientific workflows in Orlando Tools

Abstract

The paper addresses issues of continuous integration in the development of scientific applications based on workflows (special case of distributed applied software packages) for heterogeneous computing environments. The preparation and carrying out of scientific computational experiments are often accompanied by intensive software modification. Thus, there is a need for the following stages: building, testing, debugging, and installation new versions of software in heterogeneous nodes of environment. These stages may take longer time overheads than computations themselves. The solution to this challenge lies in the use of tools for continuous integration of software. However, such tools require deep integration with the tools for the workflow development because of scientific workflow specifics. To this end, the paper describes the combination of the author's Orlando Tools framework for the development and use packages with the GitLab system that is widely used for continuous integration. Such combination significantly reduces the complexity of software continuous when developing and using packages.