Delving into Transformer for Incremental Semantic Segmentation
This addresses catastrophic forgetting in incremental semantic segmentation for computer vision applications, representing an incremental improvement over existing methods.
The authors tackled catastrophic forgetting in incremental semantic segmentation by proposing TISS, a Transformer-based method with patch-wise contrastive losses, which significantly outperformed state-of-the-art methods on Pascal-VOC 2012 and ADE20K datasets.
Incremental semantic segmentation(ISS) is an emerging task where old model is updated by incrementally adding new classes. At present, methods based on convolutional neural networks are dominant in ISS. However, studies have shown that such methods have difficulty in learning new tasks while maintaining good performance on old ones (catastrophic forgetting). In contrast, a Transformer based method has a natural advantage in curbing catastrophic forgetting due to its ability to model both long-term and short-term tasks. In this work, we explore the reasons why Transformer based architecture are more suitable for ISS, and accordingly propose propose TISS, a Transformer based method for Incremental Semantic Segmentation. In addition, to better alleviate catastrophic forgetting while preserving transferability on ISS, we introduce two patch-wise contrastive losses to imitate similar features and enhance feature diversity respectively, which can further improve the performance of TISS. Under extensive experimental settings with Pascal-VOC 2012 and ADE20K datasets, our method significantly outperforms state-of-the-art incremental semantic segmentation methods.