On Connections between Regularizations for Improving DNN Robustness
It addresses the problem of enhancing DNN robustness for image classification and other applications, but is incremental as it focuses on theoretical analysis rather than new methods.
This paper analyzes regularization methods for improving adversarial robustness in deep neural networks, revealing connections between input-gradient, Jacobian, curvature, and cross-Lipschitz regularizations to reinterpret their functionality.
This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future.