DAC: Data-free Automatic Acceleration of Convolutional Networks
This addresses the problem of efficient model deployment on resource-limited devices for applications like image classification and object detection, though it is incremental as it builds on existing decomposition methods.
The paper tackles the trade-off between speed and accuracy in deploying deep learning models on mobile/IoT devices by proposing DAC, a data-free decomposition method that factorizes convolutional layers to reduce FLOPs while maintaining high accuracy, achieving reductions such as 53% FLOPs for VGG16 with a 2% accuracy drop.
Deploying a deep learning model on mobile/IoT devices is a challenging task. The difficulty lies in the trade-off between computation speed and accuracy. A complex deep learning model with high accuracy runs slowly on resource-limited devices, while a light-weight model that runs much faster loses accuracy. In this paper, we propose a novel decomposition method, namely DAC, that is capable of factorizing an ordinary convolutional layer into two layers with much fewer parameters. DAC computes the corresponding weights for the newly generated layers directly from the weights of the original convolutional layer. Thus, no training (or fine-tuning) or any data is needed. The experimental results show that DAC reduces a large number of floating-point operations (FLOPs) while maintaining high accuracy of a pre-trained model. If 2% accuracy drop is acceptable, DAC saves 53% FLOPs of VGG16 image classification model on ImageNet dataset, 29% FLOPS of SSD300 object detection model on PASCAL VOC2007 dataset, and 46% FLOPS of a multi-person pose estimation model on Microsoft COCO dataset. Compared to other existing decomposition methods, DAC achieves better performance.