Discrete Deep Feature Extraction: A Theory and New Architectures
This work provides a theoretical framework for analyzing discrete CNN architectures, which is incremental as it builds on prior continuous-time theories.
The paper tackles the mathematical analysis of discrete deep convolutional neural networks for feature extraction, establishing deformation and translation sensitivity results and investigating how input signal properties affect feature vectors, with experiments on handwritten digit classification and facial landmark detection showing complementary results.
First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made---for the continuous-time case---in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathematical framework for their analysis. Specifically, we establish deformation and translation sensitivity results of local and global nature, and we investigate how certain structural properties of the input signal are reflected in the corresponding feature vectors. Our theory applies to general filters and general Lipschitz-continuous non-linearities and pooling operators. Experiments on handwritten digit classification and facial landmark detection---including feature importance evaluation---complement the theoretical findings.