Exploring Feature Reuse in DenseNet Architectures
This work addresses efficiency improvements in neural network design for image classification, but it is incremental as it builds on existing DenseNet methods.
The paper tackled the problem of excessive connectivity in DenseNet architectures by investigating whether full connections to all previous layers are necessary for feature reuse, and found that using local dense connectivity with increased growth rates can achieve higher accuracy and more efficient feature reuse.
Densely Connected Convolutional Networks (DenseNets) have been shown to achieve state-of-the-art results on image classification tasks while using fewer parameters and computation than competing methods. Since each layer in this architecture has full access to the feature maps of all previous layers, the network is freed from the burden of having to relearn previously useful features, thus alleviating issues with vanishing gradients. In this work we explore the question: To what extent is it necessary to connect to all previous layers in order to reap the benefits of feature reuse? To this end, we introduce the notion of local dense connectivity and present evidence that less connectivity, allowing for increased growth rate at a fixed network capacity, can achieve a more efficient reuse of features and lead to higher accuracy in dense architectures.