A Dynamical System Perspective for Lipschitz Neural Networks
This work addresses the need for robust neural networks in adversarial settings, offering a novel method that improves upon previous approaches but is incremental in nature.
The paper tackled the problem of building 1-Lipschitz neural networks to enhance robustness against adversarial examples, resulting in a scalable architecture called Convex Potential Layer that provides benefits as an ℓ₂-provable defense, as demonstrated in experiments on multiple datasets.
The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the {\em Convex Potential Layer} (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an $\ell_2$-provable defense against adversarial examples.