Differentially Private Online Federated Learning with Correlated Noise
This addresses privacy concerns for distributed machine learning systems handling continuous data streams, though it appears incremental as it builds on existing DP and federated learning frameworks.
The paper tackles the problem of maintaining privacy in online federated learning with streaming non-iid data by introducing a differentially private algorithm that uses temporally correlated noise to improve utility. It establishes a dynamic regret bound under privacy constraints and demonstrates effectiveness through numerical experiments.
We introduce a novel differentially private algorithm for online federated learning that employs temporally correlated noise to enhance utility while ensuring privacy of continuously released models. To address challenges posed by DP noise and local updates with streaming non-iid data, we develop a perturbed iterate analysis to control the impact of the DP noise on the utility. Moreover, we demonstrate how the drift errors from local updates can be effectively managed under a quasi-strong convexity condition. Subject to an $(ε, δ)$-DP budget, we establish a dynamic regret bound over the entire time horizon, quantifying the impact of key parameters and the intensity of changes in dynamic environments. Numerical experiments confirm the efficacy of the proposed algorithm.