Automatic Rank Selection for High-Speed Convolutional Neural Network
This work addresses the need for efficient CNN acceleration for applications requiring high-speed processing, though it is incremental as it builds on existing low-rank decomposition techniques.
The paper tackles the problem of automatically selecting the rank for low-rank decomposition of convolutional neural networks to accelerate inference and training while maintaining accuracy, achieving faster inference and training times than state-of-the-art methods with nearly identical accuracy.
Low-rank decomposition plays a central role in accelerating convolutional neural network (CNN), and the rank of decomposed kernel-tensor is a key parameter that determines the complexity and accuracy of a neural network. In this paper, we define rank selection as a combinatorial optimization problem and propose a methodology to minimize network complexity while maintaining the desired accuracy. Combinatorial optimization is not feasible due to search space limitations. To restrict the search space and obtain the optimal rank, we define the space constraint parameters with a boundary condition. We also propose a linearly-approximated accuracy function to predict the fine-tuned accuracy of the optimized CNN model during the cost reduction. Experimental results on AlexNet and VGG-16 show that the proposed rank selection algorithm satisfies the accuracy constraint. Our method combined with truncated-SVD outperforms state-of-the-art methods in terms of inference and training time at almost the same accuracy.