Game-based scheduling algorithm to achieve optimize profit in MapReduce environment

Abstract

MapReduce is a programming model and an associated implementation for processing and generating large data sets. Providing MapReduce as a service is the development future trend. By leveraging the game theory, this paper proposes a scheduling algorithm to deal with the competition for resources between multiple jobs in MapReduce. Firstly, we present a model that could estimate job executing time, and then a utility function of job and an optimization objective are brought forward; thirdly, we present a game model to solve the optimization problem. The proof and the solution are also present. Finally, we implement the algorithm and experiment it in a hadoop cluster. The result shows the present algorithm could schedule jobs rational. © 2013 Springer-Verlag.