Robust and Efficient Collaborative Learning
This addresses the challenge of efficient and secure collaborative learning for distributed systems, offering a novel solution to reduce communication overhead.
The paper tackles the problem of adversarial behaviors in collaborative machine learning by proposing Robust Pull-based Epidemic Learning (RPEL), a scalable approach that reduces communication costs from O(n^2) to O(n log n) while maintaining robustness and competitive accuracy.
Collaborative machine learning is challenged by training-time adversarial behaviors. Existing approaches to tolerate such behaviors either rely on a central server or induce high communication costs. We propose Robust Pull-based Epidemic Learning (RPEL), a novel, scalable collaborative approach to ensure robust learning despite adversaries. RPEL does not rely on any central server and, unlike traditional methods, where communication costs grow in $\mathcal{O}(n^2)$ with the number of nodes $n$, RPEL employs a pull-based epidemic-based communication strategy that scales in $\mathcal{O}(n \log n)$. By pulling model parameters from small random subsets of nodes, RPEL significantly lowers the number of required messages without compromising convergence guarantees, which hold with high probability. Empirical results demonstrate that RPEL maintains robustness in adversarial settings, competes with all-to-all communication accuracy, and scales efficiently across large networks.