Fully Transformer Networks for Semantic Image Segmentation
This work addresses the problem of semantic image segmentation for computer vision researchers by introducing a novel pure Transformer approach, which is incremental as it builds on existing Transformer and CNN hybrid methods.
The authors tackled semantic image segmentation by proposing a pure Transformer-based framework, achieving state-of-the-art results on benchmarks like PASCAL Context, ADE20K, COCOStuff, and CelebAMask-HQ with improved performance metrics.
Transformers have shown impressive performance in various natural language processing and computer vision tasks, due to the capability of modeling long-range dependencies. Recent progress has demonstrated that combining such Transformers with CNN-based semantic image segmentation models is very promising. However, it is not well studied yet on how well a pure Transformer based approach can achieve for image segmentation. In this work, we explore a novel framework for semantic image segmentation, which is encoder-decoder based Fully Transformer Networks (FTN). Specifically, we first propose a Pyramid Group Transformer (PGT) as the encoder for progressively learning hierarchical features, meanwhile reducing the computation complexity of the standard Visual Transformer (ViT). Then, we propose a Feature Pyramid Transformer (FPT) to fuse semantic-level and spatial-level information from multiple levels of the PGT encoder for semantic image segmentation. Surprisingly, this simple baseline can achieve better results on multiple challenging semantic segmentation and face parsing benchmarks, including PASCAL Context, ADE20K, COCOStuff, and CelebAMask-HQ. The source code will be released on https://github.com/BR-IDL/PaddleViT.