Universality of Power-of-d Load Balancing in Many-Server Systems

Abstract

We consider a system of [Formula: see text] parallel single-server queues with unit exponential service rates and a single dispatcher where tasks arrive as a Poisson process of rate [Formula: see text]. When a task arrives, the dispatcher assigns it to a server with the shortest queue among [Formula: see text] randomly selected servers ([Formula: see text]). This load balancing strategy is referred to as a JSQ([Formula: see text]) scheme, noting that it subsumes the celebrated Join-the-Shortest Queue (JSQ) policy as a crucial special case for [Formula: see text]. We construct a stochastic coupling to bound the difference in the queue length processes between the JSQ policy and a JSQ([Formula: see text]) scheme with an arbitrary value of [Formula: see text]. We use the coupling to derive the fluid limit in the regime where [Formula: see text] as [Formula: see text] with [Formula: see text], along with the associated fixed point. The fluid limit turns out not to depend on the exact growth rate of [Formula: see text] and in particular coincides with that for the JSQ policy. We further leverage the coupling to establish that the diffusion limit in the critical regime where [Formula: see text] as [Formula: see text] with [Formula: see text] corresponds to that for the JSQ policy. These results indicate that the optimality of the JSQ policy can be preserved at the fluid level and diffusion level while reducing the overhead by nearly a factor O([Formula: see text]) and O([Formula: see text]), respectively.