Joint Spatial and Layer Attention for Convolutional Networks
This work addresses performance bottlenecks in computer vision tasks like camera localization and scene classification, offering incremental improvements over existing methods.
The paper tackles the problem of improving Convolutional Neural Networks by jointly attending to different layers and spatial locations, demonstrating effectiveness on camera pose regression and indoor scene classification tasks. It reduces median error by 18.8% for position and 8.2% for orientation in camera localization, and improves mean accuracy by 3.4% in scene classification.
In this paper, we propose a novel approach that learns to sequentially attend to different Convolutional Neural Networks (CNN) layers (i.e., ``what'' feature abstraction to attend to) and different spatial locations of the selected feature map (i.e., ``where'') to perform the task at hand. Specifically, at each Recurrent Neural Network (RNN) step, both a CNN layer and localized spatial region within it are selected for further processing. We demonstrate the effectiveness of this approach on two computer vision tasks: (i) image-based six degree of freedom camera pose regression and (ii) indoor scene classification. Empirically, we show that combining the ``what'' and ``where'' aspects of attention improves network performance on both tasks. We evaluate our method on standard benchmarks for camera localization (Cambridge, 7-Scenes, and TUM-LSI) and for scene classification (MIT-67 Indoor Scenes). For camera localization our approach reduces the median error by 18.8\% for position and 8.2\% for orientation (averaged over all scenes), and for scene classification it improves the mean accuracy by 3.4\% over previous methods.