Hidden Cost of Randomized Smoothing
This reveals a critical flaw in a popular robustness technique, impacting researchers and practitioners relying on randomized smoothing for secure AI systems.
The paper identifies that randomized smoothing, a method for scalable robustness guarantees in neural networks, causes shrinking decision boundaries leading to class-wise accuracy disparity, and noise augmentation in training fails to resolve this due to inconsistent objectives.
The fragility of modern machine learning models has drawn a considerable amount of attention from both academia and the public. While immense interests were in either crafting adversarial attacks as a way to measure the robustness of neural networks or devising worst-case analytical robustness verification with guarantees, few methods could enjoy both scalability and robustness guarantees at the same time. As an alternative to these attempts, randomized smoothing adopts a different prediction rule that enables statistical robustness arguments which easily scale to large networks. However, in this paper, we point out the side effects of current randomized smoothing workflows. Specifically, we articulate and prove two major points: 1) the decision boundaries of smoothed classifiers will shrink, resulting in disparity in class-wise accuracy; 2) applying noise augmentation in the training process does not necessarily resolve the shrinking issue due to the inconsistent learning objectives.