On the Effect of Defections in Federated Learning and How to Prevent Them
This addresses a critical issue in federated learning for applications where agents may withdraw, improving collaboration and model quality.
The paper tackles the problem of agents defecting in federated learning, which harms model robustness and generalization, and introduces a novel optimization algorithm with theoretical guarantees to prevent defections while ensuring convergence.
Federated learning is a machine learning protocol that enables a large population of agents to collaborate over multiple rounds to produce a single consensus model. There are several federated learning applications where agents may choose to defect permanently$-$essentially withdrawing from the collaboration$-$if they are content with their instantaneous model in that round. This work demonstrates the detrimental impact of such defections on the final model's robustness and ability to generalize. We also show that current federated optimization algorithms fail to disincentivize these harmful defections. We introduce a novel optimization algorithm with theoretical guarantees to prevent defections while ensuring asymptotic convergence to an effective solution for all participating agents. We also provide numerical experiments to corroborate our findings and demonstrate the effectiveness of our algorithm.