Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions
This work addresses a specific bottleneck in attention mechanisms for computer vision tasks, offering incremental improvements for image classification.
The authors tackled the problem of information reduction in attention mechanisms for computer vision by proposing a global attention mechanism that retains channel and spatial information, resulting in stable performance improvements on CIFAR-100 and ImageNet-1K datasets compared to recent methods.
A variety of attention mechanisms have been studied to improve the performance of various computer vision tasks. However, the prior methods overlooked the significance of retaining the information on both channel and spatial aspects to enhance the cross-dimension interactions. Therefore, we propose a global attention mechanism that boosts the performance of deep neural networks by reducing information reduction and magnifying the global interactive representations. We introduce 3D-permutation with multilayer-perceptron for channel attention alongside a convolutional spatial attention submodule. The evaluation of the proposed mechanism for the image classification task on CIFAR-100 and ImageNet-1K indicates that our method stably outperforms several recent attention mechanisms with both ResNet and lightweight MobileNet.