MicroNet: Improving Image Recognition with Extremely Low FLOPs
It addresses efficient image recognition for resource-constrained devices, offering a novel method with significant gains over prior work.
This paper tackles the problem of performance degradation in image recognition at extremely low computational costs, such as 5M FLOPs on ImageNet, by introducing MicroNet with micro-factorized convolution and Dynamic Shift Max activation, achieving 59.4% top-1 accuracy at 12M FLOPs and outperforming MobileNetV3 by 9.6%.
This paper aims at addressing the problem of substantial performance degradation at extremely low computational cost (e.g. 5M FLOPs on ImageNet classification). We found that two factors, sparse connectivity and dynamic activation function, are effective to improve the accuracy. The former avoids the significant reduction of network width, while the latter mitigates the detriment of reduction in network depth. Technically, we propose micro-factorized convolution, which factorizes a convolution matrix into low rank matrices, to integrate sparse connectivity into convolution. We also present a new dynamic activation function, named Dynamic Shift Max, to improve the non-linearity via maxing out multiple dynamic fusions between an input feature map and its circular channel shift. Building upon these two new operators, we arrive at a family of networks, named MicroNet, that achieves significant performance gains over the state of the art in the low FLOP regime. For instance, under the constraint of 12M FLOPs, MicroNet achieves 59.4\% top-1 accuracy on ImageNet classification, outperforming MobileNetV3 by 9.6\%. Source code is at \href{https://github.com/liyunsheng13/micronet}{https://github.com/liyunsheng13/micronet}.