On the Stability Analysis of Open Federated Learning Systems
This work addresses stability issues in federated learning for systems with dynamic client participation, offering a theoretical framework that is incremental but provides specific insights for such open environments.
The paper tackles the problem of convergence in open federated learning systems where clients dynamically join or leave, proposing a new stability metric to quantify model magnitude instead of fixed convergence. It theoretically derives stability radii for local SGD and Adam algorithms under strong convexity and smoothness assumptions, validated with simulations on synthetic and real-world datasets.
We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process. Given the variability of the number of present clients, convergence to a fixed model cannot be guaranteed in open systems. Instead, we resort to a new performance metric that we term the stability of open FL systems, which quantifies the magnitude of the learned model in open systems. Under the assumption that local clients' functions are strongly convex and smooth, we theoretically quantify the radius of stability for two FL algorithms, namely local SGD and local Adam. We observe that this radius relies on several key parameters, including the function condition number as well as the variance of the stochastic gradient. Our theoretical results are further verified by numerical simulations on both synthetic and real-world benchmark data-sets.