$A^2$-Nets: Double Attention Networks
This addresses the problem of inefficient long-range feature modeling in image/video recognition for researchers and practitioners, offering a plug-and-play solution with significant performance gains.
The paper tackles the inefficiency of modeling long-range relations in CNNs by introducing a double attention block that aggregates and propagates global features, enabling a ResNet-50 with this block to outperform ResNet-152 on ImageNet-1k with over 40% fewer parameters and less FLOPs, and achieve state-of-the-art results on Kinetics and UCF-101 with higher efficiency.
Learning to capture long-range relations is fundamental to image/video recognition. Existing CNN models generally rely on increasing depth to model such relations which is highly inefficient. In this work, we propose the "double attention block", a novel component that aggregates and propagates informative global features from the entire spatio-temporal space of input images/videos, enabling subsequent convolution layers to access features from the entire space efficiently. The component is designed with a double attention mechanism in two steps, where the first step gathers features from the entire space into a compact set through second-order attention pooling and the second step adaptively selects and distributes features to each location via another attention. The proposed double attention block is easy to adopt and can be plugged into existing deep neural networks conveniently. We conduct extensive ablation studies and experiments on both image and video recognition tasks for evaluating its performance. On the image recognition task, a ResNet-50 equipped with our double attention blocks outperforms a much larger ResNet-152 architecture on ImageNet-1k dataset with over 40% less the number of parameters and less FLOPs. On the action recognition task, our proposed model achieves the state-of-the-art results on the Kinetics and UCF-101 datasets with significantly higher efficiency than recent works.