On Lipschitz Bounds of General Convolutional Neural Networks
This work provides tools for analyzing the stability and discriminative power of CNNs, which is important for researchers in machine learning and neural network theory, though it is incremental as it builds on existing Lipschitz bound concepts.
The authors tackled the problem of estimating Lipschitz bounds for general convolutional neural networks (CNNs) by proposing a linear program, and they applied this to analyze networks like AlexNet and GoogleNet, establishing concentration inequalities and a nonlinear discriminant analysis for feature separation.
Many convolutional neural networks (CNNs) have a feed-forward structure. In this paper, a linear program that estimates the Lipschitz bound of such CNNs is proposed. Several CNNs, including the scattering networks, the AlexNet and the GoogleNet, are studied numerically and compared to the theoretical bounds. Next, concentration inequalities of the output distribution to a stationary random input signal expressed in terms of the Lipschitz bound are established. The Lipschitz bound is further used to establish a nonlinear discriminant analysis designed to measure the separation between features of different classes.