Jointly Computation- and Communication-Efficient Distributed Learning
This work addresses efficiency challenges in distributed learning for networked systems, though it appears incremental as it builds on existing ADMM methods with specific optimizations.
The paper tackles distributed learning over networks by proposing a novel ADMM-based algorithm that is both computation- and communication-efficient, achieving exact linear convergence in strongly convex settings and showing competitive performance in numerical comparisons.
We address distributed learning problems over undirected networks. Specifically, we focus on designing a novel ADMM-based algorithm that is jointly computation- and communication-efficient. Our design guarantees computational efficiency by allowing agents to use stochastic gradients during local training. Moreover, communication efficiency is achieved as follows: i) the agents perform multiple training epochs between communication rounds, and ii) compressed transmissions are used. We prove exact linear convergence of the algorithm in the strongly convex setting. We corroborate our theoretical results by numerical comparisons with state of the art techniques on a classification task.