Enhancing Privacy and Communication Efficiency in Federated Learning Through Selective Low-Rank Adaptation and Differential Privacy

Abstract

Federated learning (FL) enables collaborative model training without centralizing raw data, but its application to large-scale vision models remains constrained by high communication cost, data heterogeneity, and privacy risks. Furthermore, in real-world applications such as autonomous driving and healthcare, model updates can inadvertently expose sensitive information even without direct data sharing. This highlights the need for frameworks that balance privacy, efficiency, and accuracy. The current approach to addressing information exposure involves encrypting data by incorporating additional encoding. However, such approaches to encrypting data significantly increase communication costs. In this paper, we propose Federated Share-A Low-Rank Adaptation with Differential Privacy (FedSA-LoRA-DP), a parameter-efficient and privacy-preserving federated learning framework. The framework combines selective aggregation of low-rank parameters with Differential Privacy (DP), ensuring that only lightweight components are shared while formally bounding individual data influence. Since DP simply perturbs the numeric values of existing parameters without altering their dimensionality or structure, it does not increase communication cost. This design allows FedSA-LoRA-DP to provide strong privacy guarantees while maintaining communication efficiency and model accuracy. Experiments on CIFAR-100, MNIST, and SVHN datasets demonstrate that the proposed framework achieves accuracy comparable to non-private counterparts, even under heterogeneous non-independent and identically distributed data and partial client participation. These results demonstrate that integrating differential privacy into low-rank adaptation enables privacy-preserving and communication-efficient federated learning without sacrificing model performance across heterogeneous environments.