Evenly Cascaded Convolutional Networks
This addresses the need for compact, high-performance neural networks for resource-constrained applications, representing an incremental improvement in network architecture design.
The paper tackles the problem of designing efficient small neural networks by introducing the Evenly Cascaded Convolutional Network (ECN), which uses two interacting feature streams and a consistent resizing ratio to simplify architecture design. It achieves state-of-the-art results, with a 6-layered ECN under 500k parameters reaching 95.24% accuracy on CIFAR-10 and 78.99% on CIFAR-100.
We introduce Evenly Cascaded convolutional Network (ECN), a neural network taking inspiration from the cascade algorithm of wavelet analysis. ECN employs two feature streams - a low-level and high-level steam. At each layer these streams interact, such that low-level features are modulated using advanced perspectives from the high-level stream. ECN is evenly structured through resizing feature map dimensions by a consistent ratio, which removes the burden of ad-hoc specification of feature map dimensions. ECN produces easily interpretable features maps, a result whose intuition can be understood in the context of scale-space theory. We demonstrate that ECN's design facilitates the training process through providing easily trainable shortcuts. We report new state-of-the-art results for small networks, without the need for additional treatment such as pruning or compression - a consequence of ECN's simple structure and direct training. A 6-layered ECN design with under 500k parameters achieves 95.24% and 78.99% accuracy on CIFAR-10 and CIFAR-100 datasets, respectively, outperforming the current state-of-the-art on small parameter networks, and a 3 million parameter ECN produces results competitive to the state-of-the-art.