Addressing Non-IID with Data Quantity Skew in Federated Learning

Abstract

Non-IID is one of the key challenges in federated learning. Data heterogeneity may lead to slower convergence, reduced accuracy, and more training rounds. To address the common Non-IID data distribution problem in federated learning, we propose a comprehensive dynamic optimization approach based on existing methods. It leverages MAP estimation of the Dirichlet parameter (Formula presented.) to dynamically adjust the regularization coefficient (Formula presented.) and introduces orthogonal gradient coefficients (Formula presented.) to mitigate gradient interference among different classes. The approach is compatible with existing federated learning frameworks and can be easily integrated. Achieves significant accuracy improvements in both mildly and severely Non-IID scenarios while maintaining a strong performance lower bound.