Towards Flexible Device Participation in Federated Learning
This work addresses scalability and robustness issues for federated learning systems in real-world scenarios with unreliable devices, though it appears incremental as it builds on existing federated learning frameworks.
The paper tackles the problem of strict device participation requirements in federated learning by extending the paradigm to allow inactive devices, incomplete updates, and mid-training arrivals/departures, proposing a new aggregation scheme that converges under these conditions and analyzing impacts on convergence for non-IID data.
Traditional federated learning algorithms impose strict requirements on the participation rates of devices, which limit the potential reach of federated learning. This paper extends the current learning paradigm to include devices that may become inactive, compute incomplete updates, and depart or arrive in the middle of training. We derive analytical results to illustrate how allowing more flexible device participation can affect the learning convergence when data is not independently and identically distributed (non-IID). We then propose a new federated aggregation scheme that converges even when devices may be inactive or return incomplete updates. We also study how the learning process can adapt to early departures or late arrivals, and analyze their impacts on the convergence.