Nested-TNT: Hierarchical Vision Transformers with Multi-Scale Feature Processing
This work addresses a specific bottleneck in vision transformers for computer vision researchers, offering incremental improvements in image classification tasks.
The paper tackled the problem of insufficient feature processing in vision transformers by proposing Nested-TNT, a hierarchical model that incorporates multi-scale features and reduces redundancy in attention mechanisms, achieving improvements of up to 2.78% in classification accuracy on datasets like CIFAR10 and FLOWERS102 compared to prior models.
Transformer has been applied in the field of computer vision due to its excellent performance in natural language processing, surpassing traditional convolutional neural networks and achieving new state-of-the-art. ViT divides an image into several local patches, known as "visual sentences". However, the information contained in the image is vast and complex, and focusing only on the features at the "visual sentence" level is not enough. The features between local patches should also be taken into consideration. In order to achieve further improvement, the TNT model is proposed, whose algorithm further divides the image into smaller patches, namely "visual words," achieving more accurate results. The core of Transformer is the Multi-Head Attention mechanism, and traditional attention mechanisms ignore interactions across different attention heads. In order to reduce redundancy and improve utilization, we introduce the nested algorithm and apply the Nested-TNT to image classification tasks. The experiment confirms that the proposed model has achieved better classification performance over ViT and TNT, exceeding 2.25%, 1.1% on dataset CIFAR10 and 2.78%, 0.25% on dataset FLOWERS102 respectively.