Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
It is a survey paper that summarizes existing methods for improving efficiency in deep learning, making it incremental as it does not introduce new techniques.
This paper provides a comprehensive survey of recent advances in network acceleration, compression, and accelerator design for deep convolutional neural networks, addressing the challenge of high computational complexity and resource consumption for deployment in real-time or resource-limited applications.
Deep neural networks have evolved remarkably over the past few years and they are currently the fundamental tools of many intelligent systems. At the same time, the computational complexity and resource consumption of these networks also continue to increase. This will pose a significant challenge to the deployment of such networks, especially in real-time applications or on resource-limited devices. Thus, network acceleration has become a hot topic within the deep learning community. As for hardware implementation of deep neural networks, a batch of accelerators based on FPGA/ASIC have been proposed in recent years. In this paper, we provide a comprehensive survey of recent advances in network acceleration, compression and accelerator design from both algorithm and hardware points of view. Specifically, we provide a thorough analysis of each of the following topics: network pruning, low-rank approximation, network quantization, teacher-student networks, compact network design and hardware accelerators. Finally, we will introduce and discuss a few possible future directions.