Repurposing GANs for One-shot Semantic Part Segmentation
This addresses the challenge of data scarcity in segmentation tasks for computer vision researchers, though it is incremental as it adapts existing GAN methods to a new application.
The paper tackles the problem of semantic part segmentation with minimal labeled data by repurposing GANs to extract pixel-wise representations, achieving results comparable to supervised baselines that use many more labels.
While GANs have shown success in realistic image generation, the idea of using GANs for other tasks unrelated to synthesis is underexplored. Do GANs learn meaningful structural parts of objects during their attempt to reproduce those objects? In this work, we test this hypothesis and propose a simple and effective approach based on GANs for semantic part segmentation that requires as few as one label example along with an unlabeled dataset. Our key idea is to leverage a trained GAN to extract pixel-wise representation from the input image and use it as feature vectors for a segmentation network. Our experiments demonstrate that GANs representation is "readily discriminative" and produces surprisingly good results that are comparable to those from supervised baselines trained with significantly more labels. We believe this novel repurposing of GANs underlies a new class of unsupervised representation learning that is applicable to many other tasks. More results are available at https://repurposegans.github.io/.