Distributed Convex Optimization With Limited Communications
This addresses the problem of large-scale distributed optimization for scenarios with constrained communication, offering a practical solution but is incremental relative to existing methods.
The paper tackles distributed convex optimization with limited communication by proposing the DCDA algorithm, which restricts coordinate communication over time and achieves bounded convergence rates under various protocols and network architectures, with numerical simulations provided.
In this paper, a distributed convex optimization algorithm, termed \emph{distributed coordinate dual averaging} (DCDA) algorithm, is proposed. The DCDA algorithm addresses the scenario of a large distributed optimization problem with limited communication among nodes in the network. Currently known distributed subgradient methods, such as the distributed dual averaging or the distributed alternating direction method of multipliers algorithms, assume that nodes can exchange messages of large cardinality. Such network communication capabilities are not valid in many scenarios of practical relevance. In the DCDA algorithm, on the other hand, communication of each coordinate of the optimization variable is restricted over time. For the proposed algorithm, we bound the rate of convergence under different communication protocols and network architectures. We also consider the extensions to the case of imperfect gradient knowledge and the case in which transmitted messages are corrupted by additive noise or are quantized. Relevant numerical simulations are also provided.