Local adapt-then-combine algorithms for distributed nonsmooth optimization: Achieving provable communication acceleration
This work addresses communication bottlenecks in distributed optimization for networked systems, offering a novel theoretical framework that is incremental but provides strong specific gains.
The paper tackles distributed composite optimization over networks by proposing FlexATC, a communication-efficient Adapt-Then-Combine framework that unifies existing algorithms and achieves provable communication acceleration. It establishes sublinear and linear convergence rates for convex and strongly convex settings, with the linear rate decoupled from objective functions and network topology, allowing communication to be skipped in most iterations without rate deterioration.
This paper is concerned with the distributed composite optimization problem over networks, where agents aim to minimize a sum of local smooth components and a common nonsmooth term. Leveraging the probabilistic local updates mechanism, we propose a communication-efficient Adapt-Then-Combine (ATC) framework, FlexATC, unifying numerous ATC-based distributed algorithms. Under stepsizes independent of the network topology and the number of local updates, we establish sublinear and linear convergence rates for FlexATC in convex and strongly convex settings, respectively. Remarkably, in the strong convex setting, the linear rate is decoupled from the objective functions and network topology, and FlexATC permits communication to be skipped in most iterations without any deterioration of the linear rate. In addition, the proposed unified theory demonstrates for the first time that local updates provably lead to communication acceleration for ATC-based distributed algorithms. Numerical experiments further validate the efficacy of the proposed framework and corroborate the theoretical results.