INSIDE: Steering Spatial Attention with Non-Imaging Information in CNNs
This work addresses the challenge of incorporating spatial dependencies in convolutional networks for medical imaging segmentation, offering a domain-specific solution for tasks like cardiac or lesion analysis.
The authors tackled the problem of integrating non-imaging information into segmentation networks to improve performance by proposing INSIDE, a method that enables spatial localization conditioned on factors like lesion location or cardiac phase, resulting in improved segmentation accuracy on datasets such as CLEVR-Seg and ACDC.
We consider the problem of integrating non-imaging information into segmentation networks to improve performance. Conditioning layers such as FiLM provide the means to selectively amplify or suppress the contribution of different feature maps in a linear fashion. However, spatial dependency is difficult to learn within a convolutional paradigm. In this paper, we propose a mechanism to allow for spatial localisation conditioned on non-imaging information, using a feature-wise attention mechanism comprising a differentiable parametrised function (e.g. Gaussian), prior to applying the feature-wise modulation. We name our method INstance modulation with SpatIal DEpendency (INSIDE). The conditioning information might comprise any factors that relate to spatial or spatio-temporal information such as lesion location, size, and cardiac cycle phase. Our method can be trained end-to-end and does not require additional supervision. We evaluate the method on two datasets: a new CLEVR-Seg dataset where we segment objects based on location, and the ACDC dataset conditioned on cardiac phase and slice location within the volume. Code and the CLEVR-Seg dataset are available at https://github.com/jacenkow/inside.