A game theory based mapreduce scheduling algorithm

Abstract

A Hadoop MapReduce cluster is an environment where multi-users, multi-jobs and multi-tasks share the same physical resources. Because of the competitive relationship among the jobs, we need to select the most suitable job to be sent to the cluster. In this paper we consider this problem as a two-level scheduling problem based on a detailed cost model. Then we abstract these scheduling problems into two games. And we solve these games in using some methods of game theory to achieve the solution. Our strategy improves the utilization efficiency of each type of the resources. And it can also avoid the unnecessary transmission of data. © 2013 Springer Science+Business Media New York.