MimiC: Combating Client Dropouts in Federated Learning by Mimicking Central Updates
This addresses a critical challenge for federated learning in mobile edge networks, where unpredictable client availability can degrade training, and it is an incremental improvement over existing methods.
The paper tackled the problem of client dropouts in federated learning, which hinders convergence, by proposing MimiC, a novel algorithm that modifies model updates to mimic central updates, resulting in stable convergence and better model performance than baselines.
Federated learning (FL) is a promising framework for privacy-preserving collaborative learning, where model training tasks are distributed to clients and only the model updates need to be collected at a server. However, when being deployed at mobile edge networks, clients may have unpredictable availability and drop out of the training process, which hinders the convergence of FL. This paper tackles such a critical challenge. Specifically, we first investigate the convergence of the classical FedAvg algorithm with arbitrary client dropouts. We find that with the common choice of a decaying learning rate, FedAvg oscillates around a stationary point of the global loss function, which is caused by the divergence between the aggregated and desired central update. Motivated by this new observation, we then design a novel training algorithm named MimiC, where the server modifies each received model update based on the previous ones. The proposed modification of the received model updates mimics the imaginary central update irrespective of dropout clients. The theoretical analysis of MimiC shows that divergence between the aggregated and central update diminishes with proper learning rates, leading to its convergence. Simulation results further demonstrate that MimiC maintains stable convergence performance and learns better models than the baseline methods.