HOTCAKE: Higher Order Tucker Articulated Kernels for Deeper CNN Compression
This work addresses the need for lightweight neural networks in edge computing, offering an incremental improvement over existing compression methods.
The paper tackles the problem of compressing convolutional neural networks (CNNs) for edge computing by proposing HOTCAKE, a higher-order Tucker decomposition scheme, which achieves state-of-the-art compression with minimal accuracy loss.
The emerging edge computing has promoted immense interests in compacting a neural network without sacrificing much accuracy. In this regard, low-rank tensor decomposition constitutes a powerful tool to compress convolutional neural networks (CNNs) by decomposing the 4-way kernel tensor into multi-stage smaller ones. Building on top of Tucker-2 decomposition, we propose a generalized Higher Order Tucker Articulated Kernels (HOTCAKE) scheme comprising four steps: input channel decomposition, guided Tucker rank selection, higher order Tucker decomposition and fine-tuning. By subjecting each CONV layer to HOTCAKE, a highly compressed CNN model with graceful accuracy trade-off is obtained. Experiments show HOTCAKE can compress even pre-compressed models and produce state-of-the-art lightweight networks.