On the Stability of Deep Networks
This provides foundational insights into the stability and properties of deep networks, which is important for researchers in machine learning theory.
The paper tackles the problem of understanding deep neural networks with random weights by formally proving that they perform a distance-preserving embedding of data, with implications for training data size and network structure.
In this work we study the properties of deep neural networks (DNN) with random weights. We formally prove that these networks perform a distance-preserving embedding of the data. Based on this we then draw conclusions on the size of the training data and the networks' structure. A longer version of this paper with more results and details can be found in (Giryes et al., 2015). In particular, we formally prove in the longer version that DNN with random Gaussian weights perform a distance-preserving embedding of the data, with a special treatment for in-class and out-of-class data.