Two-layer accumulated quantized compression for communication-efficient federated learning: TLAQC

2Citations
Citations of this article
8Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

Federated learning enables multiple nodes to perform local computations and collaborate to complete machine learning tasks without centralizing private data of nodes. However, the frequent model gradients upload/download operations required by the framework result in high communication costs, which have become the main bottleneck for federated learning as deep models scale up, hindering its performance. In this paper, we propose a two-layer accumulated quantized compression algorithm (TLAQC) that effectively reduces the communication cost of federated learning. TLAQC achieves this by reducing both the cost of individual communication and the number of global communication rounds. TLAQC introduces a revised quantization method called RQSGD, which employs zero-value correction to mitigate ineffective quantization phenomena and minimize average quantization errors. Additionally, TLAQC reduces the frequency of gradient information uploads through an adaptive threshold and parameter self-inspection mechanism, further reducing communication costs. It also accumulates quantization errors and retained weight deltas to compensate for gradient knowledge loss. Through quantization correction and two-layer accumulation, TLAQC significantly reduces precision loss caused by communication compression. Experimental results demonstrate that RQSGD achieves an incidence of ineffective quantization as low as 0.003% and reduces the average quantization error to 1.6 × 10 - 5 . Compared to full-precision FedAVG, TLAQC compresses uploaded traffic to only 6.73% while increasing accuracy by 1.25%.

Cite

CITATION STYLE

APA

Ren, Y., Cao, Y., Ye, C., & Cheng, X. (2023). Two-layer accumulated quantized compression for communication-efficient federated learning: TLAQC. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-38916-x

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free