Rega-Net:Retina Gabor Attention for Deep Convolutional Neural Networks
This work addresses a specific bottleneck in CNN attention mechanisms for computer vision tasks, offering incremental improvements in accuracy.
The paper tackled the problem of limited receptive fields in attention mechanisms for CNNs by proposing Rega-Net, a novel attention method inspired by the human retina, which achieved 79.96% top-1 accuracy on ImageNet-1K and 43.1% mAP on COCO2017, with up to a 3.5% mAP increase over baselines.
Extensive research works demonstrate that the attention mechanism in convolutional neural networks (CNNs) effectively improves accuracy. Nevertheless, few works design attention mechanisms using large receptive fields. In this work, we propose a novel attention method named Rega-net to increase CNN accuracy by enlarging the receptive field. Inspired by the mechanism of the human retina, we design convolutional kernels to resemble the non-uniformly distributed structure of the human retina. Then, we sample variable-resolution values in the Gabor function distribution and fill these values in retina-like kernels. This distribution allows essential features to be more visible in the center position of the receptive field. We further design an attention module including these retina-like kernels. Experiments demonstrate that our Rega-Net achieves 79.96% top-1 accuracy on ImageNet-1K classification and 43.1% mAP on COCO2017 object detection. The mAP of the Rega-Net increased by up to 3.5% compared to baseline networks.