AttentionRNN: A Structured Spatial Attention Mechanism
This addresses the issue of inconsistent attention masks in multi-modal learning tasks for researchers and practitioners in computer vision and deep learning, representing a novel method rather than an incremental improvement.
The paper tackled the problem of prior attention frameworks lacking explicit structural dependencies among attention variables, which hindered consistent attention mask prediction, by developing a novel structured spatial attention mechanism that enforces structure through sequential prediction in bi-directional raster-scan orders, resulting in consistent quantitative and qualitative improvements on tasks like image categorization, question answering, and image generation.
Visual attention mechanisms have proven to be integrally important constituent components of many modern deep neural architectures. They provide an efficient and effective way to utilize visual information selectively, which has shown to be especially valuable in multi-modal learning tasks. However, all prior attention frameworks lack the ability to explicitly model structural dependencies among attention variables, making it difficult to predict consistent attention masks. In this paper we develop a novel structured spatial attention mechanism which is end-to-end trainable and can be integrated with any feed-forward convolutional neural network. This proposed AttentionRNN layer explicitly enforces structure over the spatial attention variables by sequentially predicting attention values in the spatial mask in a bi-directional raster-scan and inverse raster-scan order. As a result, each attention value depends not only on local image or contextual information, but also on the previously predicted attention values. Our experiments show consistent quantitative and qualitative improvements on a variety of recognition tasks and datasets; including image categorization, question answering and image generation.