On the Robustness of Distributed Machine Learning against Transfer Attacks
This addresses security vulnerabilities in distributed ML systems for practitioners, though it is incremental as it builds on prior work on robustness.
The paper tackles the problem of robustness in distributed machine learning against transfer attacks by examining fully heterogeneous models across both training and inference phases, showing improvements such as a 40% increase in robust accuracy on CIFAR10 with minimal impact on clean accuracy.
Although distributed machine learning (distributed ML) is gaining considerable attention in the community, prior works have independently looked at instances of distributed ML in either the training or the inference phase. No prior work has examined the combined robustness stemming from distributing both the learning and the inference process. In this work, we explore, for the first time, the robustness of distributed ML models that are fully heterogeneous in training data, architecture, scheduler, optimizer, and other model parameters. Supported by theory and extensive experimental validation using CIFAR10 and FashionMNIST, we show that such properly distributed ML instantiations achieve across-the-board improvements in accuracy-robustness tradeoffs against state-of-the-art transfer-based attacks that could otherwise not be realized by current ensemble or federated learning instantiations. For instance, our experiments on CIFAR10 show that for the Common Weakness attack, one of the most powerful state-of-the-art transfer-based attacks, our method improves robust accuracy by up to 40%, with a minimal impact on clean task accuracy.