Decentralized SGD with Over-the-Air Computation
This work addresses communication bottlenecks in decentralized machine learning for wireless networks, though it is incremental as it builds on existing DSGD and over-the-air computation methods.
The paper tackles the problem of decentralized stochastic gradient descent (DSGD) in noisy wireless networks by proposing two transmission schemes, OAC-P2P and OAC-MAC, and shows that OAC-MAC achieves better convergence with fewer communication rounds in MNIST image classification tasks.
We study the performance of decentralized stochastic gradient descent (DSGD) in a wireless network, where the nodes collaboratively optimize an objective function using their local datasets. Unlike the conventional setting, where the nodes communicate over error-free orthogonal communication links, we assume that transmissions are prone to additive noise and interference.We first consider a point-to-point (P2P) transmission strategy, termed the OAC-P2P scheme, in which the node pairs are scheduled in an orthogonal fashion to minimize interference. Since in the DSGD framework, each node requires a linear combination of the neighboring models at the consensus step, we then propose the OAC-MAC scheme, which utilizes the signal superposition property of the wireless medium to achieve over-the-air computation (OAC). For both schemes, we cast the scheduling problem as a graph coloring problem. We numerically evaluate the performance of these two schemes for the MNIST image classification task under various network conditions. We show that the OAC-MAC scheme attains better convergence performance with a fewer communication rounds.