Decentralized Learning over Wireless Networks: The Effect of Broadcast with Random Access
This work addresses communication efficiency for decentralized learning systems in wireless environments, but it is incremental as it builds on existing D-SGD methods.
The paper tackled the problem of communication bottlenecks in decentralized learning over wireless networks by analyzing the impact of broadcast transmission and random access policies on D-SGD convergence, showing that optimizing access probability to maximize successful links accelerates convergence.
In this work, we focus on the communication aspect of decentralized learning, which involves multiple agents training a shared machine learning model using decentralized stochastic gradient descent (D-SGD) over distributed data. In particular, we investigate the impact of broadcast transmission and probabilistic random access policy on the convergence performance of D-SGD, considering the broadcast nature of wireless channels and the link dynamics in the communication topology. Our results demonstrate that optimizing the access probability to maximize the expected number of successful links is a highly effective strategy for accelerating the system convergence.