Towards Federated Learning on Time-Evolving Heterogeneous Data
This addresses the challenge of realistic, dynamic data scenarios in federated learning for privacy-preserving edge computing, though it appears incremental as it builds on existing FL methods.
The paper tackles the problem of federated learning with time-evolving heterogeneous data, such as changing client data or intermittent participation, by proposing Continual Federated Learning (CFL), which theoretically shows faster convergence than FedAvg and significantly outperforms other state-of-the-art baselines in experiments.
Federated Learning (FL) is a learning paradigm that protects privacy by keeping client data on edge devices. However, optimizing FL in practice can be difficult due to the diversity and heterogeneity of the learning system. Despite recent research efforts to improve the optimization of heterogeneous data, the impact of time-evolving heterogeneous data in real-world scenarios, such as changing client data or intermittent clients joining or leaving during training, has not been studied well. In this work, we propose Continual Federated Learning (CFL), a flexible framework for capturing the time-evolving heterogeneity of FL. CFL can handle complex and realistic scenarios, which are difficult to evaluate in previous FL formulations, by extracting information from past local data sets and approximating local objective functions. We theoretically demonstrate that CFL methods have a faster convergence rate than FedAvg in time-evolving scenarios, with the benefit depending on approximation quality. Through experiments, we show that our numerical findings match the convergence analysis and that CFL methods significantly outperform other state-of-the-art FL baselines.