Self-Supervised Vision Transformers Are Efficient Segmentation Learners for Imperfect Labels
This provides a cost-effective solution for semantic segmentation tasks where obtaining perfect labels is expensive or impractical.
The study tackled semantic segmentation with imperfect labels by using self-supervised vision transformers, achieving a 11.5% performance gain over baselines under zero-shot vision-language-model-based labels.
This study demonstrates a cost-effective approach to semantic segmentation using self-supervised vision transformers (SSVT). By freezing the SSVT backbone and training a lightweight segmentation head, our approach effectively utilizes imperfect labels, thereby improving robustness to label imperfections. Empirical experiments show significant performance improvements over existing methods for various annotation types, including scribble, point-level, and image-level labels. The research highlights the effectiveness of self-supervised vision transformers in dealing with imperfect labels, providing a practical and efficient solution for semantic segmentation while reducing annotation costs. Through extensive experiments, we confirm that our method outperforms baseline models for all types of imperfect labels. Especially under the zero-shot vision-language-model-based label, our model exhibits 11.5\%p performance gain compared to the baseline.