Lipschitz Networks and Distributional Robustness
This work addresses robustness and generalization issues in machine learning models, particularly for deep neural networks, by providing a theoretical framework to interpret and quantify distributional robustness.
The paper tackled the problem of quantifying distributional robustness in deep neural networks by bounding the distributionally robust risk with a regularized empirical risk involving the model's Lipschitz constant, and showed that this risk upper-bounds adversarial training risk.
Robust risk minimisation has several advantages: it has been studied with regards to improving the generalisation properties of models and robustness to adversarial perturbation. We bound the distributionally robust risk for a model class rich enough to include deep neural networks by a regularised empirical risk involving the Lipschitz constant of the model. This allows us to interpretand quantify the robustness properties of a deep neural network. As an application we show the distributionally robust risk upperbounds the adversarial training risk.