Convolutional Rectangular Attention Module
This provides a novel spatial attention mechanism for convolutional models, improving performance and interpretability, but it is incremental as it builds on existing attention methods.
The paper tackles the problem of irregular boundaries in spatial attention maps by introducing a rectangular attention module that constrains the attention region to a rectangle parametrized by 5 parameters, resulting in better stability and generalization, and it systematically outperforms position-wise counterparts in experiments.
In this paper, we introduce a novel spatial attention module that can be easily integrated to any convolutional network. This module guides the model to pay attention to the most discriminative part of an image. This enables the model to attain a better performance by an end-to-end training. In conventional approaches, a spatial attention map is typically generated in a position-wise manner. Thus, it is often resulting in irregular boundaries and so can hamper generalization to new samples. In our method, the attention region is constrained to be rectangular. This rectangle is parametrized by only 5 parameters, allowing for a better stability and generalization to new samples. In our experiments, our method systematically outperforms the position-wise counterpart. So that, we provide a novel useful spatial attention mechanism for convolutional models. Besides, our module also provides the interpretability regarding the \textit{where to look} question, as it helps to know the part of the input on which the model focuses to produce the prediction.