Harmonic Networks: Integrating Spectral Information into CNNs
This work addresses the need for integrating spectral information into CNNs for image classification, but it is incremental as it builds on existing architectures with modest gains.
The paper tackles the problem of capturing spectral information in CNNs by proposing harmonic blocks that replace conventional convolutional layers with learned combinations of spectral filters, resulting in comparable or better performance on classification tasks for small NORB, CIFAR10, and CIFAR100 datasets.
Convolutional neural networks (CNNs) learn filters in order to capture local correlation patterns in feature space. In contrast, in this paper we propose harmonic blocks that produce features by learning optimal combinations of spectral filters defined by the Discrete Cosine Transform. The harmonic blocks are used to replace conventional convolutional layers to construct partial or fully harmonic CNNs. We extensively validate our approach and show that the introduction of harmonic blocks into state-of-the-art CNN baseline architectures results in comparable or better performance in classification tasks on small NORB, CIFAR10 and CIFAR100 datasets.