UniNeXt: Exploring A Unified Architecture for Vision Recognition
This work addresses a foundational issue in computer vision by enhancing backbone architectures, potentially benefiting researchers and practitioners in vision tasks, though it appears incremental as it builds on existing spatial token mixers.
The authors tackled the problem of suboptimal general architectures in vision backbones by proposing UniNeXt, a unified architecture that improved performance across various spatial token mixers, including achieving state-of-the-art results with naive local window attention.
Vision Transformers have shown great potential in computer vision tasks. Most recent works have focused on elaborating the spatial token mixer for performance gains. However, we observe that a well-designed general architecture can significantly improve the performance of the entire backbone, regardless of which spatial token mixer is equipped. In this paper, we propose UniNeXt, an improved general architecture for the vision backbone. To verify its effectiveness, we instantiate the spatial token mixer with various typical and modern designs, including both convolution and attention modules. Compared with the architecture in which they are first proposed, our UniNeXt architecture can steadily boost the performance of all the spatial token mixers, and narrows the performance gap among them. Surprisingly, our UniNeXt equipped with naive local window attention even outperforms the previous state-of-the-art. Interestingly, the ranking of these spatial token mixers also changes under our UniNeXt, suggesting that an excellent spatial token mixer may be stifled due to a suboptimal general architecture, which further shows the importance of the study on the general architecture of vision backbone.