Transformer Scale Gate for Semantic Segmentation
This addresses the need for more accurate semantic segmentation in computer vision applications, though it appears incremental as it builds on existing hierarchical vision Transformer architectures.
The paper tackles the problem of sub-optimal multi-scale feature combination in transformer-based semantic segmentation models by proposing a Transformer Scale Gate (TSG) module that selects optimal scales using attention cues, achieving consistent gains on Pascal Context and ADE20K datasets.
Effectively encoding multi-scale contextual information is crucial for accurate semantic segmentation. Existing transformer-based segmentation models combine features across scales without any selection, where features on sub-optimal scales may degrade segmentation outcomes. Leveraging from the inherent properties of Vision Transformers, we propose a simple yet effective module, Transformer Scale Gate (TSG), to optimally combine multi-scale features.TSG exploits cues in self and cross attentions in Vision Transformers for the scale selection. TSG is a highly flexible plug-and-play module, and can easily be incorporated with any encoder-decoder-based hierarchical vision Transformer architecture. Extensive experiments on the Pascal Context and ADE20K datasets demonstrate that our feature selection strategy achieves consistent gains.