An Adaptive Model Averaging Procedure for Federated Learning (AdaFed)

Abstract

Federated Learning (FL) is an enabling technology for Machine Learning in scenarios in which it is impossible, for privacy and/or regulatory reasons, to analyze data in a centralized manner. FL envisages that distributed clients cooperate to learn a model without any data exchange, in favor of a model averaging procedure that is coordinated by a server. In this work, we present the Adaptive Federated Learning (AdaFed) algorithm, that extends the original Federated Averaging algorithm by: (i) dynamically weighting the local models, based on their performance, for the averaging procedure; (ii) adapting the loss function at every communication round depending on the training behavior. This work specializes AdaFed for both classification and regression tasks, and reports several validation tests on benchmarking dataset, showing its enhanced robustness against unbalanced data distributions and adversarial clients.