CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction
This provides a more efficient and lightweight solution for visual tasks on small-scale datasets, though it is incremental as it builds on existing CNN and transformer methods.
The paper tackled the problem of inefficient aggregation of CNNs and vision transformers for multi-scale feature extraction by developing CTA-Net, which achieved a TOP-1 accuracy of 86.76% with 20.32M parameters and 2.83B FLOPs on small-scale datasets.
Convolutional neural networks (CNNs) and vision transformers (ViTs) have become essential in computer vision for local and global feature extraction. However, aggregating these architectures in existing methods often results in inefficiencies. To address this, the CNN-Transformer Aggregation Network (CTA-Net) was developed. CTA-Net combines CNNs and ViTs, with transformers capturing long-range dependencies and CNNs extracting localized features. This integration enables efficient processing of detailed local and broader contextual information. CTA-Net introduces the Light Weight Multi-Scale Feature Fusion Multi-Head Self-Attention (LMF-MHSA) module for effective multi-scale feature integration with reduced parameters. Additionally, the Reverse Reconstruction CNN-Variants (RRCV) module enhances the embedding of CNNs within the transformer architecture. Extensive experiments on small-scale datasets with fewer than 100,000 samples show that CTA-Net achieves superior performance (TOP-1 Acc 86.76\%), fewer parameters (20.32M), and greater efficiency (FLOPs 2.83B), making it a highly efficient and lightweight solution for visual tasks on small-scale datasets (fewer than 100,000).