Convolutional Neural Networks as 2-D systems
This provides a new theoretical framework for analyzing CNNs, which could benefit researchers in machine learning and control theory by improving robustness verification, though it appears incremental in its application to existing methods.
The paper tackles the problem of representing convolutional neural networks (CNNs) as 2-D dynamical systems, resulting in a novel 2-D Lur'e system perspective that enables more efficient Lipschitz constant estimation using robust control theory.
This paper introduces a novel representation of convolutional Neural Networks (CNNs) in terms of 2-D dynamical systems. To this end, the usual description of convolutional layers with convolution kernels, i.e., the impulse responses of linear filters, is realized in state space as a linear time-invariant 2-D system. The overall convolutional Neural Network composed of convolutional layers and nonlinear activation functions is then viewed as a 2-D version of a Lur'e system, i.e., a linear dynamical system interconnected with static nonlinear components. One benefit of this 2-D Lur'e system perspective on CNNs is that we can use robust control theory much more efficiently for Lipschitz constant estimation than previously possible.