Connection Reduction of DenseNet for Image Recognition
This work addresses inefficiencies in neural network architectures for image recognition on small datasets, offering incremental improvements in speed and accuracy.
The paper tackled the problem of dense connections in CNNs being unsuitable for small datasets by proposing two new connection algorithms, ShortNet1 and ShortNet2, which reduced test error by 5% and sped up inference by 25-40% on CIFAR-10 and SVHN.
Convolutional Neural Networks (CNN) increase depth by stacking convolutional layers, and deeper network models perform better in image recognition. Empirical research shows that simply stacking convolutional layers does not make the network train better, and skip connection (residual learning) can improve network model performance. For the image classification task, models with global densely connected architectures perform well in large datasets like ImageNet, but are not suitable for small datasets such as CIFAR-10 and SVHN. Different from dense connections, we propose two new algorithms to connect layers. Baseline is a densely connected network, and the networks connected by the two new algorithms are named ShortNet1 and ShortNet2 respectively. The experimental results of image classification on CIFAR-10 and SVHN show that ShortNet1 has a 5% lower test error rate and 25% faster inference time than Baseline. ShortNet2 speeds up inference time by 40% with less loss in test accuracy. Code and pre-trained models are available at https://github.com/RuiyangJu/Connection_Reduction.