Labels4Free: Unsupervised Segmentation using StyleGAN
This addresses the problem of segmentation without labeled data for generative models, offering an incremental advance in unsupervised methods.
The paper tackles unsupervised segmentation for StyleGAN-generated objects by augmenting StyleGAN2 with a segmentation branch and splitting it into foreground and background networks, achieving results comparable to supervised state-of-the-art and clear improvements over the best unsupervised approach.
We propose an unsupervised segmentation framework for StyleGAN generated objects. We build on two main observations. First, the features generated by StyleGAN hold valuable information that can be utilized towards training segmentation networks. Second, the foreground and background can often be treated to be largely independent and be composited in different ways. For our solution, we propose to augment the StyleGAN2 generator architecture with a segmentation branch and to split the generator into a foreground and background network. This enables us to generate soft segmentation masks for the foreground object in an unsupervised fashion. On multiple object classes, we report comparable results against state-of-the-art supervised segmentation networks, while against the best unsupervised segmentation approach we demonstrate a clear improvement, both in qualitative and quantitative metrics.