Discovering Latent Classes for Semi-Supervised Semantic Segmentation
This work addresses the bottleneck of annotation costs for semantic segmentation, offering a practical solution for researchers and practitioners in computer vision.
The paper tackles the problem of high annotation costs in semantic segmentation by proposing a semi-supervised method that learns latent classes to leverage unlabeled images, achieving state-of-the-art results on Pascal VOC and Cityscapes datasets.
High annotation costs are a major bottleneck for the training of semantic segmentation systems. Therefore, methods working with less annotation effort are of special interest. This paper studies the problem of semi-supervised semantic segmentation. This means that only a small subset of the training images is annotated while the other training images do not contain any annotation. In order to leverage the information present in the unlabeled images, we propose to learn a second task that is related to semantic segmentation but easier. On labeled images, we learn latent classes consistent with semantic classes so that the variety of semantic classes assigned to a latent class is as low as possible. On unlabeled images, we predict a probability map for latent classes and use it as a supervision signal to learn semantic segmentation. The latent classes, as well as the semantic classes, are simultaneously predicted by a two-branch network. In our experiments on Pascal VOC and Cityscapes, we show that the latent classes learned this way have an intuitive meaning and that the proposed method achieves state of the art results for semi-supervised semantic segmentation.