Regulating Workers in Federated Learning by Yardstick Competition

Abstract

Due to the large size of the training data, distributed learning approaches such as federated learning have gained attention recently. However, the convergence rate of distributed learning suffers from heterogeneous worker performance. In this paper, we consider an incentive mechanism for workers to mitigate the delays in completion of each batch. To motivate the workers to perform at their best by assigning higher computational resources to the learning task, we use a yardstick of average desired delay to complete each mini-batch calculation. The rewards are determined by how much each worker deviates from this yardstick. We analytically obtain the optimum equilibrium strategy of the workers as well as the optimal reward function of the model owner that achieves the average desired delay while minimizing the cost of operation. Our numerical results indicate that by adjusting budget parameters, the model owner should judiciously decide on the number of workers due to trade off between the diversity provided by the number of workers and the latency of completing the training.