Consistency Regularization for Certified Robustness of Smoothed Classifiers
This work addresses adversarial robustness for machine learning models, offering an efficient and effective solution that is incremental but impactful in improving certified defenses.
The paper tackles the trade-off between accuracy and certified robustness in smoothed classifiers by introducing consistency regularization over noise, which dramatically improves certified ℓ₂-robustness with less training cost and hyperparameters compared to state-of-the-art methods.
A recent technique of randomized smoothing has shown that the worst-case (adversarial) $\ell_2$-robustness can be transformed into the average-case Gaussian-robustness by "smoothing" a classifier, i.e., by considering the averaged prediction over Gaussian noise. In this paradigm, one should rethink the notion of adversarial robustness in terms of generalization ability of a classifier under noisy observations. We found that the trade-off between accuracy and certified robustness of smoothed classifiers can be greatly controlled by simply regularizing the prediction consistency over noise. This relationship allows us to design a robust training objective without approximating a non-existing smoothed classifier, e.g., via soft smoothing. Our experiments under various deep neural network architectures and datasets show that the "certified" $\ell_2$-robustness can be dramatically improved with the proposed regularization, even achieving better or comparable results to the state-of-the-art approaches with significantly less training costs and hyperparameters.