Shapp: Workload Management System for Massive Distributed Calculations

Abstract

The paper presents an overview of existing workload management systems and diagnoses difficulties in their use. Such systems require a complex configuration, and the process of ordering tasks to be carried out is burdened with many restrictions. A new solution is presented which supports the use of massive distributed computations. The work presents the process of designing, implementing and testing the workload management Shapp library, based on the HTCondor system. It implements a convenient application interface in the form of a dynamically linked library, which extends the capabilities of existing applications with a convenient mechanism allowing the use of massive distributed processing. Recursive computations in tree-like structure are possible using Shapp library.