Efficient Convolutional Network Learning using Parametric Log based Dual-Tree Wavelet ScatterNet
This work addresses training inefficiencies in CNNs for computer vision tasks, but it is incremental as it builds on existing ScatterNet and wavelet transform methods.
The authors tackled the problem of inefficient training in convolutional neural networks by replacing early layers with a parametric log-based dual-tree wavelet ScatterNet to extract edge-based invariant representations, resulting in improved learning efficiency and competitive performance on CIFAR-10 and Caltech-101 datasets.
We propose a DTCWT ScatterNet Convolutional Neural Network (DTSCNN) formed by replacing the first few layers of a CNN network with a parametric log based DTCWT ScatterNet. The ScatterNet extracts edge based invariant representations that are used by the later layers of the CNN to learn high-level features. This improves the training of the network as the later layers can learn more complex patterns from the start of learning because the edge representations are already present. The efficient learning of the DTSCNN network is demonstrated on CIFAR-10 and Caltech-101 datasets. The generic nature of the ScatterNet front-end is shown by an equivalent performance to pre-trained CNN front-ends. A comparison with the state-of-the-art on CIFAR-10 and Caltech-101 datasets is also presented.