iiANET: Inception Inspired Attention Hybrid Network for efficient Long-Range Dependency
This work addresses a key problem in computer vision for researchers and practitioners by proposing an incremental improvement in hybrid models for better long-range dependency modeling.
The paper tackled the challenge of efficiently combining convolutional neural networks and vision transformers to capture long-range dependencies in complex images, resulting in iiANET, a hybrid visual backbone that achieved improved performance over state-of-the-art models on various benchmarks.
The recent emergence of hybrid models has introduced a transformative approach to computer vision, gradually moving beyond conventional convolutional neural net-works and vision transformers. However, efficiently combining these two paradigms to better capture long-range dependencies in complex images remains a challenge. In this paper, we present iiANET (Inception Inspired Attention Network), an efficient hybrid visual backbone designed to improve the modeling of long-range dependen-cies. The core innovation of iiANET is the iiABlock, a unified building block that in-tegrates global r-MHSA (Multi-Head Self-Attention) and convolutional layers in paral-lel. This design enables iiABlock to simultaneously capture global context and local details, making it highly effective for extracting rich and diverse features. By effi-ciently fusing these complementary representations, iiABlock allows iiANET to achieve strong feature interaction while maintaining computational efficiency. Exten-sive qualitative and quantitative evaluations across various benchmarks show im-proved performance over several state-of-the-art models.