RecConv: Efficient Recursive Convolutions for Multi-Frequency Representations
This work addresses the problem of scaling receptive fields efficiently for researchers and practitioners in computer vision, offering an incremental improvement over existing convolution methods.
The paper tackles the efficiency and optimization challenges of large-kernel convolutions in vision transformers by introducing RecConv, a recursive decomposition strategy that uses small-kernel convolutions to achieve multi-frequency representations with linear parameter growth and constant FLOPs, resulting in RecNeXt-M3 outperforming RepViT-M1.1 by 1.9 AP^{box} on COCO with similar FLOPs.
Recent advances in vision transformers (ViTs) have demonstrated the advantage of global modeling capabilities, prompting widespread integration of large-kernel convolutions for enlarging the effective receptive field (ERF). However, the quadratic scaling of parameter count and computational complexity (FLOPs) with respect to kernel size poses significant efficiency and optimization challenges. This paper introduces RecConv, a recursive decomposition strategy that efficiently constructs multi-frequency representations using small-kernel convolutions. RecConv establishes a linear relationship between parameter growth and decomposing levels which determines the effective receptive field $k\times 2^\ell$ for a base kernel $k$ and $\ell$ levels of decomposition, while maintaining constant FLOPs regardless of the ERF expansion. Specifically, RecConv achieves a parameter expansion of only $\ell+2$ times and a maximum FLOPs increase of $5/3$ times, compared to the exponential growth ($4^\ell$) of standard and depthwise convolutions. RecNeXt-M3 outperforms RepViT-M1.1 by 1.9 $AP^{box}$ on COCO with similar FLOPs. This innovation provides a promising avenue towards designing efficient and compact networks across various modalities. Codes and models can be found at https://github.com/suous/RecNeXt.