Phocas: dimensional Byzantine-resilient stochastic gradient descent
This addresses the issue of secure distributed machine learning for systems vulnerable to adversarial attacks, representing an incremental improvement in Byzantine resilience.
The paper tackles the problem of Byzantine failures in distributed synchronous Stochastic Gradient Descent (SGD) by proposing a novel robust aggregation rule, and it shows that the technique outperforms current approaches in realistic use cases and attack scenarios.
We propose a novel robust aggregation rule for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience of the proposed aggregation rules. Empirical analysis shows that the proposed techniques outperform current approaches for realistic use cases and Byzantine attack scenarios.