Rethinking Layer-wise Feature Amounts in Convolutional Neural Network Architectures
This work addresses a fundamental design problem in CNN architecture for image classification, suggesting incremental improvements over standard practices.
The paper challenges the assumption that convolutional neural networks should have monotonically increasing feature counts per layer, finding that architectures with larger early layers achieve better accuracy on MNIST, Fashion-MNIST, and CIFAR-10 benchmarks.
We characterize convolutional neural networks with respect to the relative amount of features per layer. Using a skew normal distribution as a parametrized framework, we investigate the common assumption of monotonously increasing feature-counts with higher layers of architecture designs. Our evaluation on models with VGG-type layers on the MNIST, Fashion-MNIST and CIFAR-10 image classification benchmarks provides evidence that motivates rethinking of our common assumption: architectures that favor larger early layers seem to yield better accuracy.