Communication-Efficient Distributed Asynchronous ADMM
This addresses communication costs for large-scale federated learning applications, but is incremental as it builds on existing ADMM methods.
The paper tackled the communication bottleneck in asynchronous ADMM for distributed optimization and federated learning by introducing coarse quantization to reduce overhead, and experimentally verified convergence with neural networks.
In distributed optimization and federated learning, asynchronous alternating direction method of multipliers (ADMM) serves as an attractive option for large-scale optimization, data privacy, straggler nodes and variety of objective functions. However, communication costs can become a major bottleneck when the nodes have limited communication budgets or when the data to be communicated is prohibitively large. In this work, we propose introducing coarse quantization to the data to be exchanged in aynchronous ADMM so as to reduce communication overhead for large-scale federated learning and distributed optimization applications. We experimentally verify the convergence of the proposed method for several distributed learning tasks, including neural networks.