Dynamic Federated Learning
This work addresses the challenge of dynamic data drifts in federated learning for multi-agent systems, but it is incremental as it builds on existing static analyses.
The paper tackles the problem of federated learning in non-stationary environments where data drifts over time, establishing that performance depends on data variability, model variability, and a tracking term inversely related to learning rate, clarifying trade-offs between convergence and tracking.
Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. While many federated learning architectures process data in an online manner, and are hence adaptive by nature, most performance analyses assume static optimization problems and offer no guarantees in the presence of drifts in the problem solution or data characteristics. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm. The results clarify the trade-off between convergence and tracking performance.