A Hybrid Scattering Transform for Signals with Isolated Singularities
This work addresses a specific bottleneck in neural network design for signal processing, but it is incremental as it builds on existing scattering transform theory.
The paper tackles the problem of determining appropriate filters for later layers in convolutional neural networks by proposing a two-layer hybrid scattering transform that combines wavelet and Gabor filters to characterize signals with isolated singularities, showing that Gabor measurements can synthesize sparse signals.
The scattering transform is a wavelet-based model of Convolutional Neural Networks originally introduced by S. Mallat. Mallat's analysis shows that this network has desirable stability and invariance guarantees and therefore helps explain the observation that the filters learned by early layers of a Convolutional Neural Network typically resemble wavelets. Our aim is to understand what sort of filters should be used in the later layers of the network. Towards this end, we propose a two-layer hybrid scattering transform. In our first layer, we convolve the input signal with a wavelet filter transform to promote sparsity, and, in the second layer, we convolve with a Gabor filter to leverage the sparsity created by the first layer. We show that these measurements characterize information about signals with isolated singularities. We also show that the Gabor measurements used in the second layer can be used to synthesize sparse signals such as those produced by the first layer.