Harmonic Networks with Limited Training Samples
This work addresses the challenge of data scarcity in image processing for researchers and practitioners, but it is incremental as it builds on existing preset filter strategies.
The paper tackles the problem of training convolutional neural networks with limited labeled data by proposing a harmonic block using DCT filters, and it shows that this approach performs comparably to scattering networks with wavelet filters in experiments.
Convolutional neural networks (CNNs) are very popular nowadays for image processing. CNNs allow one to learn optimal filters in a (mostly) supervised machine learning context. However this typically requires abundant labelled training data to estimate the filter parameters. Alternative strategies have been deployed for reducing the number of parameters and / or filters to be learned and thus decrease overfitting. In the context of reverting to preset filters, we propose here a computationally efficient harmonic block that uses Discrete Cosine Transform (DCT) filters in CNNs. In this work we examine the performance of harmonic networks in limited training data scenario. We validate experimentally that its performance compares well against scattering networks that use wavelets as preset filters.