Lipschitz constant estimation for 1D convolutional neural networks
This work addresses the need for robust and efficient Lipschitz constant estimation in neural networks, which is crucial for stability and generalization in applications like signal processing, but it is incremental as it builds on existing dissipativity theory for a specific network type.
The paper tackles the problem of estimating Lipschitz constants for 1D convolutional neural networks by proposing a dissipativity-based method that analyzes layers using incremental quadratic constraints and solves a semidefinite program, resulting in bounds that are advantageous in accuracy and scalability as shown in examples.
In this work, we propose a dissipativity-based method for Lipschitz constant estimation of 1D convolutional neural networks (CNNs). In particular, we analyze the dissipativity properties of convolutional, pooling, and fully connected layers making use of incremental quadratic constraints for nonlinear activation functions and pooling operations. The Lipschitz constant of the concatenation of these mappings is then estimated by solving a semidefinite program which we derive from dissipativity theory. To make our method as efficient as possible, we exploit the structure of convolutional layers by realizing these finite impulse response filters as causal dynamical systems in state space and carrying out the dissipativity analysis for the state space realizations. The examples we provide show that our Lipschitz bounds are advantageous in terms of accuracy and scalability.