Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
This addresses the data scarcity issue in semantic segmentation for computer vision applications, but it is incremental as it builds on existing GAN and semi-supervised methods.
The paper tackles the problem of semantic segmentation requiring large pixel-level annotated data by proposing a semi-supervised framework using Generative Adversarial Networks (GANs) to leverage unlabeled or weakly labeled data and generated images, achieving competitive performance on datasets like PASCAL, SiftFLow, Stanford, and CamVid.
Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs significant number of pixellevel annotated data, which is often unavailable. To address this lack, in this paper, we leverage, on one hand, massive amount of available unlabeled or weakly labeled data, and on the other hand, non-real images created through Generative Adversarial Networks. In particular, we propose a semi-supervised framework ,based on Generative Adversarial Networks (GANs), which consists of a generator network to provide extra training examples to a multi-class classifier, acting as discriminator in the GAN framework, that assigns sample a label y from the K possible classes or marks it as a fake sample (extra class). The underlying idea is that adding large fake visual data forces real samples to be close in the feature space, enabling a bottom-up clustering process, which, in turn, improves multiclass pixel classification. To ensure higher quality of generated images for GANs with consequent improved pixel classification, we extend the above framework by adding weakly annotated data, i.e., we provide class level information to the generator. We tested our approaches on several challenging benchmarking visual datasets, i.e. PASCAL, SiftFLow, Stanford and CamVid, achieving competitive performance also compared to state-of-the-art semantic segmentation method