MGIC: Multigrid-in-Channels Neural Network Architectures
This work provides an incremental improvement in CNN efficiency for researchers and practitioners by reducing parameter count without sacrificing accuracy, particularly for applications sensitive to model size.
This paper introduces Multigrid-in-Channels (MGIC) neural network architectures to address the quadratic parameter growth with respect to the number of channels in standard CNNs. By replacing CNN blocks with MGIC counterparts that use nested grouped convolutions, the architectures achieve linear scaling with network width while maintaining full channel coupling, leading to reduced parameters and similar or better accuracy across image classification, segmentation, and point cloud classification tasks.
We present a multigrid-in-channels (MGIC) approach that tackles the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs). Thereby our approach addresses the redundancy in CNNs that is also exposed by the recent success of lightweight CNNs. Lightweight CNNs can achieve comparable accuracy to standard CNNs with fewer parameters; however, the number of weights still scales quadratically with the CNN's width. Our MGIC architectures replace each CNN block with an MGIC counterpart that utilizes a hierarchy of nested grouped convolutions of small group size to address this. Hence, our proposed architectures scale linearly with respect to the network's width while retaining full coupling of the channels as in standard CNNs. Our extensive experiments on image classification, segmentation, and point cloud classification show that applying this strategy to different architectures like ResNet and MobileNetV3 reduces the number of parameters while obtaining similar or better accuracy.