Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
This work exposes vulnerabilities in Byzantine-tolerant SGD defenses, which is critical for securing distributed machine learning systems against malicious actors.
The paper tackles the problem of Byzantine failures in distributed machine learning by breaking two robust aggregation methods, coordinate-wise median and Krum, using new attack strategies based on inner product manipulation, with theoretical proofs and empirical validation.
Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily, including malicious and compromised workers. In this paper, we break two prevailing Byzantine-tolerant techniques. Specifically we show robust aggregation methods for synchronous SGD -- coordinate-wise median and Krum -- can be broken using new attack strategies based on inner product manipulation. We prove our results theoretically, as well as show empirical validation.