Load-aware dynamic replication management in a Data Grid

Abstract

Data Grids are increasingly popular in novel, demanding and data-intensive eScience applications. In these applications, vast amounts of data, generated by specialized instruments, need to be collaboratively accessed, processed and analyzed by a large number of users spread across several organizations. The nearly unlimited storage capabilities of Data Grids allow these data to be replicated at different sites in order to guarantee a high degree of availability. For updateable data objects, several replicas per object need to be maintained in an eager way. In addition, read-only copies serve users' needs of data with different levels of freshness. The number of updateable replicas has to be dynamically adapted to optimize the trade-off between synchronization overhead and the gain which can be achieved by balancing the load of update transactions. Due to the particular characteristics of the Grid, especially due to the absence of a global coordinator, replication management needs to be provided in a completely distributed way. This includes the synchronization of concurrent updates as well as the dynamic deployment and undeployment of replicas based on actual access characteristics which might change over time. In this paper we present the Re:GRIDiT approach to dynamic replica deployment and undeployment in the Grid. Based on a combination of local load statistics, proximity and data access patterns, Re:GRIDiT dynamically adds new replicas or removes existing ones without impacting global correctness. In addition, we provide a detailed evaluation of the overall performance of the dynamic Re:GRIDiT protocol which shows increased throughput with respect to the replication management protocol with a static number of replicas. © Springer-Verlag 2009.