Deep Learning by Scattering
This provides a mathematical framework for deep learning, but it appears incremental as it builds on existing scattering transform concepts.
The paper tackles the problem of modeling deep neural networks with l2 pooling by introducing general scattering transforms, showing that unsupervised learning can be optimized for space contraction to preserve volume in unlabeled examples, and supervised classification uses averaged scattering for multiple classes.
We introduce general scattering transforms as mathematical models of deep neural networks with l2 pooling. Scattering networks iteratively apply complex valued unitary operators, and the pooling is performed by a complex modulus. An expected scattering defines a contractive representation of a high-dimensional probability distribution, which preserves its mean-square norm. We show that unsupervised learning can be casted as an optimization of the space contraction to preserve the volume occupied by unlabeled examples, at each layer of the network. Supervised learning and classification are performed with an averaged scattering, which provides scattering estimations for multiple classes.