Online Multi-Resource Social Welfare Maximization for Non-Preemptive Jobs

Abstract

Multi-resource allocation is ubiquitous in datacenters. Usually, the datacenter scheduler associates each user with a utility function and then allocates multiple resources so as to maximize the social welfare, which is the sum of the utilities of the users. We refer to this problem as the Social Welfare Maximization (SWM) problem. Meanwhile, given the dynamic nature of datacenters, the scheduler is supposed to support non-preemptive scheduling, where jobs of users that are being processed would not be interrupted by the reconfiguration of resources. Non-preemptive solutions to SWM incorporating pricing considerations include dynamic pricing and bidding/auctions. However, both strategies are quite complex. Meanwhile, traditional Lagrange solutions to SWM does not support non-preemptive job scheduling. In this paper, we propose Adaptive Dominant Resource Fairness (ADRF), that provides near-optimal social welfare where each user is assigned an \alpha -fair utility function and when 1 < \alpha \leq 2. Based on ADRF, we propose Cumulative ADRF (C-ADRF), an online algorithm that greedily assigns the idle resources to the current poorest user, making it quite simple to schedule jobs from users non-preemptively. Compared with the polynomial or exponential form of time complexities in other SWM solutions, C-ADRF requires only O(\log N) work, where N is the number of users. Extensive simulations using Google and Facebook cluster-data traces show an impressive behavior of C-ADRF in providing social welfare, only 3% and 2% away from the optimal social welfare, respectively.