Provenance trails in the Wings/Pegasus system

Abstract

Our research focuses on creating and executing large-scale scientific workflows that often involve thousands of computations over distributed, shared resources. We describe an approach to workflow creation and refinement that uses semantic representations to (1) describe complex scientific applications in a data-independent manner, (2) automatically generate workflows of computations for given data sets, and (3) map the workflows to available computing resources for efficient execution. Our approach is implemented in the Wings/Pegasus workflow system and has been demonstrated in a variety of scientific application domains. This paper illustrates the application-level provenance information generated Wings during workflow creation and the refinement provenance by the Pegasus mapping system for execution over grid computing environments. We show how this information is used in answering the queries of the First Provenance Challenge. Copyright © 2007 John Wiley & Sons, Ltd.