Quantized and Asynchronous Federated Learning
This addresses scalability and efficiency issues in federated learning for distributed systems, though it is incremental as it builds on existing asynchronous methods by adding quantization.
The paper tackles the communication bottleneck in asynchronous federated learning by introducing a quantized algorithm (QAFeL) with a hidden-state scheme to prevent error propagation, achieving an optimal O(1/√T) convergence rate on non-convex objectives without requiring bounded gradients or uniform client arrivals.
Recent advances in federated learning have shown that asynchronous variants can be faster and more scalable than their synchronous counterparts. However, their design does not include quantization, which is necessary in practice to deal with the communication bottleneck. To bridge this gap, we develop a novel algorithm, Quantized Asynchronous Federated Learning (QAFeL), which introduces a hidden-state quantization scheme to avoid the error propagation caused by direct quantization. QAFeL also includes a buffer to aggregate client updates, ensuring scalability and compatibility with techniques such as secure aggregation. Furthermore, we prove that QAFeL achieves an $\mathcal{O}(1/\sqrt{T})$ ergodic convergence rate for stochastic gradient descent on non-convex objectives, which is the optimal order of complexity, without requiring bounded gradients or uniform client arrivals. We also prove that the cross-term error between staleness and quantization only affects the higher-order error terms. We validate our theoretical findings on standard benchmarks.