Laplacian Networks: Bounding Indicator Function Smoothness for Neural Network Robustness
This addresses robustness issues in sensitive applications like vision systems, but it is incremental as it builds upon existing regularization methods.
The authors tackled the problem of improving deep neural network robustness against adversarial attacks and other distortions by proposing a new regularizer based on Laplacian similarity graphs, which enforces smooth class boundaries and demonstrated effectiveness on vision datasets.
For the past few years, Deep Neural Network (DNN) robustness has become a question of paramount importance. As a matter of fact, in sensitive settings misclassification can lead to dramatic consequences. Such misclassifications are likely to occur when facing adversarial attacks, hardware failures or limitations, and imperfect signal acquisition. To address this question, authors have proposed different approaches aiming at increasing the robustness of DNNs, such as adding regularizers or training using noisy examples. In this paper we propose a new regularizer built upon the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DNN architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. Since it is agnostic to the type of deformations that are expected when predicting with the DNN, the proposed regularizer can be combined with existing ad-hoc methods. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness of DNNs on classical supervised learning vision datasets.