Online Learning with Adversaries: A Differential-Inclusion Analysis
This addresses the challenge of robust and asynchronous federated learning for distributed systems, though it appears incremental as it builds on existing FL methods with a new convergence proof.
The paper tackles the problem of fully asynchronous online federated learning with adversaries by proposing an observation-matrix-based framework, and it demonstrates that the algorithm almost surely converges to the desired mean, making it the first asynchronous FL method with such a guarantee in adversarial settings.
We introduce an observation-matrix-based framework for fully asynchronous online Federated Learning (FL) with adversaries. In this work, we demonstrate its effectiveness in estimating the mean of a random vector. Our main result is that the proposed algorithm almost surely converges to the desired mean $μ.$ This makes ours the first asynchronous FL method to have an a.s. convergence guarantee in the presence of adversaries. We derive this convergence using a novel differential-inclusion-based two-timescale analysis. Two other highlights of our proof include (a) the use of a novel Lyapunov function to show that $μ$ is the unique global attractor for our algorithm's limiting dynamics, and (b) the use of martingale and stopping-time theory to show that our algorithm's iterates are almost surely bounded.