Convolutional Neural Networks Do Work with Pre-Defined Filters
This work addresses the challenge of high computational costs in CNNs for researchers and practitioners, though it is incremental as it builds on existing depthwise convolution methods.
The paper tackles the problem of reducing the parameter complexity in Convolutional Neural Networks by introducing Pre-defined Filter Convolutional Neural Networks (PFCNNs), where convolution kernels are fixed during training, and demonstrates their effectiveness on multiple datasets like Caltech101 and CIFAR10.
We present a novel class of Convolutional Neural Networks called Pre-defined Filter Convolutional Neural Networks (PFCNNs), where all nxn convolution kernels with n>1 are pre-defined and constant during training. It involves a special form of depthwise convolution operation called a Pre-defined Filter Module (PFM). In the channel-wise convolution part, the 1xnxn kernels are drawn from a fixed pool of only a few (16) different pre-defined kernels. In the 1x1 convolution part linear combinations of the pre-defined filter outputs are learned. Despite this harsh restriction, complex and discriminative features are learned. These findings provide a novel perspective on the way how information is processed within deep CNNs. We discuss various properties of PFCNNs and prove their effectiveness using the popular datasets Caltech101, CIFAR10, CUB-200-2011, FGVC-Aircraft, Flowers102, and Stanford Cars. Our implementation of PFCNNs is provided on Github https://github.com/Criscraft/PredefinedFilterNetworks