Workspace - a Scientific Workflow System for enabling Research Impact

Abstract

Scientific Workflow Systems (SWSs) have become an essential platform in many areas of research. Compared to writing research software from scratch, they provide a more accessible way to acquire data, customise inputs and combine algorithms in order to produce research outputs that are reproducible. Today there are a number of SWSs such as Apache Taverna, Galaxy, Kepler, KNIME, Pegasus and CSIRO’s Workspace. Depending on your definition you may also consider environments such as MATLAB or programming languages with dedicated scientific support, such as Python with its SciPy and NumPy libraries, to be SWSs. All address different subsets of requirements, but generally attempt to address at least the following four to varying degrees: 1) improving researcher productivity, 2) providing the ability to create reproducible workflows, 3) enabling collaboration between different research teams, and 4) providing some aspect of portability and interoperability - either between different sciences or computing environments, or both. Most SWSs provide a generic set of capabilities such as a graphical tool to construct, save, load, and edit workflows, and a set of well proven functions, such as file I/O, and an execution environment. Workspace is an SWS that has been under continuous development at CSIRO since 2005. Its development is guided by four core themes; Analyse, Collaborate, Commercialise and Everywhere. Three of these map to the SWS requirements: • Analyse - improving researcher productivity and enabling reproducibility by enabling a higher level of reuse and sharing of software components • Collaborate - enabling collaboration between different research teams by providing a common platform and interoperability between otherwise incompatible software elements • Everywhere - the ability to build multiplatform applications leveraging a range of computing and communication platforms and to support transdisciplinary knowledge integration Workspace’s fourth core theme of Commercialise is a key differentiator to most other SWSs and one not usually at the forefront of a researcher’s mind. Workspace has been developed with a goal to shorten the path from research to impact, which in many cases takes the form of translating research into usable, robust, standalone tools for research, industry and commercial partners. Workspace provides significant productivity gains with a learning curve suited to a broad range of users, without requiring software engineering expertise. This is aided by an intuitive user interface and comprehensive help system, to produce workflows and applications. Key features include: • Easily extendible plugin architecture which allows individuals and teams to easily add their own data types, algorithms and user interface components into the framework to use and share with others. This feature has been used to expose a number of popular scientific libraries (such as OpenCV, PCL and VTK) and languages (such as Python, R and MATLAB) in the framework. • Flexible and powerful execution system enabling continuous, inline interaction with data throughout the workflow while it is running, facilitating rapid prototyping. Parallel and distributed execution of workflows is supported as are other execution models such as batch runs from a command line or embedding workflows as shared library calls from within other software. • Highly optimised and customisable real time interactive 2D plotting and 3D visualisation, combined with strong point cloud, surface and volume geometry pre and post processing capabilities. • Capability to package workflows along with plugins and user interfaces as distributable, standalone software applications. This represents a fast and cost-effective path to the commercialisation of research outputs. This paper discusses SWSs and their requirements and then introduces Workspace. Finally a high level comparison of the more popular platforms is given.