Rate-Adapted Decentralized Learning over Wireless Networks

Abstract

This paper proposes a communication strategy for decentralized learning in wireless systems that employs adaptive modulation and coding capability. The main objective of this work is to address a critical issue in decentralized learning based on the cooperative stochastic gradient descent (C-SGD) over wireless systems: the relationship between the transmission rate and the network density influences the runtime performance of learning. We first present that a dense network topology does not necessarily benefit the iteration performance of learning than a sparse one. However, it tends to degrade the runtime performance because the dense network topology requires a low-rate transmission. Based on these findings, a communication strategy is proposed in which each node optimizes its transmission rate to minimize communication time during the C-SGD under the constraints of network density. We perform numerical simulations of an image classification task under both independent and identically distributed (i.i.d.) and non-i.i.d. settings. The simulation results reveal that the preferred setting for the network density depends on the channel conditions and the biases in the training samples. Furthermore, numerical simulations of an automatic modulation classification task indicate that the preferred setting is almost the same even if the training task is different.