Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries
This addresses security and reliability issues in collaborative decision-making systems for applications like autonomous vehicles or healthcare, representing a foundational advance in robust federated learning.
The paper tackles the problem of adversarial agents in Federated Reinforcement Learning by proposing a robust policy gradient method, achieving the first global convergence guarantees with optimal sample complexity of order Ψ(1/(Nε^2) * (1 + f^2/N)), where N is the total agents and f is adversarial agents.
Federated Reinforcement Learning (FRL) allows multiple agents to collaboratively build a decision making policy without sharing raw trajectories. However, if a small fraction of these agents are adversarial, it can lead to catastrophic results. We propose a policy gradient based approach that is robust to adversarial agents which can send arbitrary values to the server. Under this setting, our results form the first global convergence guarantees with general parametrization. These results demonstrate resilience with adversaries, while achieving optimal sample complexity of order $\tilde{\mathcal{O}}\left( \frac{1}{Nε^2} \left( 1+ \frac{f^2}{N}\right)\right)$, where $N$ is the total number of agents and $f<N/2$ is the number of adversarial agents.