MFI-ResNet: Efficient ResNet Architecture Optimization via MeanFlow Compression and Selective Incubation
This work addresses the need for more efficient deep learning models in computer vision, offering a novel method that bridges generative and discriminative learning, though it is incremental in its application to existing ResNet frameworks.
The paper tackles the problem of optimizing ResNet architectures for better parameter efficiency and performance by proposing MFI-ResNet, which uses MeanFlow compression and selective incubation to reduce parameters by over 45% compared to ResNet-50 while slightly improving accuracy on CIFAR-10 and CIFAR-100 datasets.
ResNet has achieved tremendous success in computer vision through its residual connection mechanism. ResNet can be viewed as a discretized form of ordinary differential equations (ODEs). From this perspective, the multiple residual blocks within a single ResNet stage essentially perform multi-step discrete iterations of the feature transformation for that stage. The recently proposed flow matching model, MeanFlow, enables one-step generative modeling by learning the mean velocity field to transform distributions. Inspired by this, we propose MeanFlow-Incubated ResNet (MFI-ResNet), which employs a compression-expansion strategy to jointly improve parameter efficiency and discriminative performance. In the compression phase, we simplify the multi-layer structure within each ResNet stage to one or two MeanFlow modules to construct a lightweight meta model. In the expansion phase, we apply a selective incubation strategy to the first three stages, expanding them to match the residual block configuration of the baseline ResNet model, while keeping the last stage in MeanFlow form, and fine-tune the incubated model. Experimental results show that on CIFAR-10 and CIFAR-100 datasets, MFI-ResNet achieves remarkable parameter efficiency, reducing parameters by 46.28% and 45.59% compared to ResNet-50, while still improving accuracy by 0.23% and 0.17%, respectively. This demonstrates that generative flow-fields can effectively characterize the feature transformation process in ResNet, providing a new perspective for understanding the relationship between generative modeling and discriminative learning.