Learning Efficient Convolutional Networks through Irregular Convolutional Kernels
This addresses the challenge of model size for low-power device deployment, though it is incremental as it builds on existing parameter pruning techniques.
The paper tackles the problem of deploying deep neural networks on low-power devices with limited memory by introducing a method that transforms traditional square convolutional kernels into line segments, resulting in a 69% reduction in parameters and 59% reduction in calculations on DenseNet-40 with less than 2% accuracy loss.
As deep neural networks are increasingly used in applications suited for low-power devices, a fundamental dilemma becomes apparent: the trend is to grow models to absorb increasing data that gives rise to memory intensive; however low-power devices are designed with very limited memory that can not store large models. Parameters pruning is critical for deep model deployment on low-power devices. Existing efforts mainly focus on designing highly efficient structures or pruning redundant connections for networks. They are usually sensitive to the tasks or relay on dedicated and expensive hashing storage strategies. In this work, we introduce a novel approach for achieving a lightweight model from the views of reconstructing the structure of convolutional kernels and efficient storage. Our approach transforms a traditional square convolution kernel to line segments, and automatically learn a proper strategy for equipping these line segments to model diverse features. The experimental results indicate that our approach can massively reduce the number of parameters (pruned 69% on DenseNet-40) and calculations (pruned 59% on DenseNet-40) while maintaining acceptable performance (only lose less than 2% accuracy).