Lightweight Residual Densely Connected Convolutional Neural Network
This addresses the need for efficient neural networks for embedded and mobile devices, though it is incremental as it builds on existing architectures like residual and densely connected networks.
The paper tackles the problem of designing efficient convolutional neural networks for limited-resource devices by proposing lightweight residual densely connected blocks, which reduce parameters and computational operations while achieving competitive accuracy, such as state-of-the-art results on Fashion MNIST and comparable performance to models like CondenseNet and ShuffleNet.
Extremely efficient convolutional neural network architectures are one of the most important requirements for limited-resource devices (such as embedded and mobile devices). The computing power and memory size are two important constraints of these devices. Recently, some architectures have been proposed to overcome these limitations by considering specific hardware-software equipment. In this paper, the lightweight residual densely connected blocks are proposed to guaranty the deep supervision, efficient gradient flow, and feature reuse abilities of convolutional neural network. The proposed method decreases the cost of training and inference processes without using any special hardware-software equipment by just reducing the number of parameters and computational operations while achieving a feasible accuracy. Extensive experimental results demonstrate that the proposed architecture is more efficient than the AlexNet and VGGNet in terms of model size, required parameters, and even accuracy. The proposed model has been evaluated on the ImageNet, MNIST, Fashion MNIST, SVHN, CIFAR-10, and CIFAR-100. It achieves state-of-the-art results on Fashion MNIST dataset and reasonable results on the others. The obtained results show the superiority of the proposed method to efficient models such as the SqueezNet. It is also comparable with state-of-the-art efficient models such as CondenseNet and ShuffleNet.