Dynamic virtual worker nodes in a production grid

Abstract

There is a growing body of opinion that virtual machines (VMs) provide a good environment for executing user jobs on Grid compute nodes. Sites which execute jobs in specially-created virtual machines can provide levels of isolation and customisation that are unobtainable when jobs run directly on the hardware. Various solutions have been proposed for initiating and controlling such dynamic virtual environments, but issues of integration with a production Grid middleware stack have not received much attention. In addition, solutions proposed to date often require significant user involvement in the process of locating and initiating VMs. We outline a scheme for transparently providing dynamically-instantiated VM-based worker nodes in the EGEE production grid. By extending server-side software, the use of virtual machines is made invisible to the user. Users simply specify the details of their required execution environment in the standard job description language. Resource brokers then locate sites that advertise support for that particular environment. Sites that support dynamic virtual worker nodes advertise support for the various environments that they know how to create; the site's compute element is responsible for instantiating a VM that conforms to the environment description requested and for executing the job in that VM's context. We also evaluate the VM management tools available to implement such a scheme and describe their possible integration with LCG and gLite middleware. © Springer-Verlag 2006.