IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
This work addresses the need for efficient deep neural networks for applications like mobile and embedded systems, but it is incremental as it builds on existing design patterns.
The paper tackled the problem of building lightweight and efficient convolutional neural networks by combining low-rank and sparse kernels, resulting in IGCV3, which outperformed state-of-the-art models like IGCV2 and MobileNetV2 on image classification tasks (CIFAR, ImageNet) and object detection (COCO).
In this paper, we are interested in building lightweight and efficient convolutional neural networks. Inspired by the success of two design patterns, composition of structured sparse kernels, e.g., interleaved group convolutions (IGC), and composition of low-rank kernels, e.g., bottle-neck modules, we study the combination of such two design patterns, using the composition of structured sparse low-rank kernels, to form a convolutional kernel. Rather than introducing a complementary condition over channels, we introduce a loose complementary condition, which is formulated by imposing the complementary condition over super-channels, to guide the design for generating a dense convolutional kernel. The resulting network is called IGCV3. We empirically demonstrate that the combination of low-rank and sparse kernels boosts the performance and the superiority of our proposed approach to the state-of-the-arts, IGCV2 and MobileNetV2 over image classification on CIFAR and ImageNet and object detection on COCO.