Generalized Byzantine-tolerant SGD
This work addresses security vulnerabilities in distributed machine learning systems for applications requiring robustness against malicious attacks, representing an incremental improvement over existing methods.
The paper tackles the problem of securing distributed synchronous Stochastic Gradient Descent (SGD) against general Byzantine failures where attackers can arbitrarily manipulate data in parameter server architectures, proposing three new robust aggregation rules and demonstrating their Byzantine resilience properties with empirical results showing they outperform current approaches in realistic scenarios.
We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience properties of these aggregation rules. Empirical analysis shows that the proposed techniques outperform current approaches for realistic use cases and Byzantine attack scenarios.