Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation
This work offers incremental theoretical improvements for researchers in federated learning, addressing scalability issues in cross-device settings.
The paper tackles the problem of scalability in federated learning by analyzing the FedBuff algorithm under more realistic unbounded gradient assumptions, and it provides theoretical convergence rates accounting for data heterogeneity, batch size, and delays.
Synchronous updates may compromise the efficiency of cross-device federated learning once the number of active clients increases. The \textit{FedBuff} algorithm (Nguyen et al., 2022) alleviates this problem by allowing asynchronous updates (staleness), which enhances the scalability of training while preserving privacy via secure aggregation. We revisit the \textit{FedBuff} algorithm for asynchronous federated learning and extend the existing analysis by removing the boundedness assumptions from the gradient norm. This paper presents a theoretical analysis of the convergence rate of this algorithm when heterogeneity in data, batch size, and delay are considered.