Aligning model outputs for class imbalanced non-IID federated learning

Abstract

Federated Learning (FL) aims to generate a global shared model via collaborating with decentralized edge computing devices with privacy considerations. A significant challenge in FL is the non-IID data partitioning across heterogeneous devices, making the local update diverge a lot and difficult to aggregate. This diversity in local models is caused by the different posterior probability of samples when class distribution skews. Meanwhile, FL often faces imbalanced global data in practical scenarios. By analyzing the relationship between the samples’ posterior probability in different data distributions, we propose a statistically principled probability-corrected loss to align the posterior probability when models are trained on heterogeneous clients. Additionally, we share fixed prototypes on each client to constrain the distribution of heterogeneous clients’ features. Our approach can well handle non-IID FL with balanced and imbalanced global data. We combine our approach with existing FL algorithms and investigate it on common FL benchmarks. Abundant experimental results verify the superiorities of our methods.