Towards Efficient Convolutional Network Models with Filter Distribution Templates
This work addresses efficiency improvements in convolutional network design for computer vision applications, though it appears incremental as it builds on existing architectures.
The authors challenged the common practice of increasing filters in deeper layers of convolutional networks by proposing a small set of templates that modify filter distributions in VGG and ResNet architectures, resulting in models with fewer parameters and memory requirements on datasets like CIFAR, CINIC10, and TinyImagenet.
Increasing number of filters in deeper layers when feature maps are decreased is a widely adopted pattern in convolutional network design. It can be found in classical CNN architectures and in automatic discovered models. Even CNS methods commonly explore a selection of multipliers derived from this pyramidal pattern. We defy this practice by introducing a small set of templates consisting of easy to implement, intuitive and aggressive variations of the original pyramidal distribution of filters in VGG and ResNet architectures. Experiments on CIFAR, CINIC10 and TinyImagenet datasets show that models produced by our templates, are more efficient in terms of fewer parameters and memory needs.