Learning to Exploit the Prior Network Knowledge for Weakly-Supervised Semantic Segmentation
This addresses the challenge of reducing annotation costs for semantic segmentation, which is incremental as it builds on prior knowledge without external algorithms.
The paper tackles the problem of weakly-supervised semantic segmentation by learning from image labels only, using prior network knowledge as an attention mechanism to generate class-specific masks, and achieves results that outperform recent methods on the PASCAL VOC 2012 dataset.
Training a Convolutional Neural Network (CNN) for semantic segmentation typically requires to collect a large amount of accurate pixel-level annotations, a hard and expensive task. In contrast, simple image tags are easier to gather. With this paper we introduce a novel weakly-supervised semantic segmentation model able to learn from image labels, and just image labels. Our model uses the prior knowledge of a network trained for image recognition, employing these image annotations as an attention mechanism to identify semantic regions in the images. We then present a methodology that builds accurate class-specific segmentation masks from these regions, where neither external objectness nor saliency algorithms are required. We describe how to incorporate this mask generation strategy into a fully end-to-end trainable process where the network jointly learns to classify and segment images. Our experiments on PASCAL VOC 2012 dataset show that exploiting these generated class-specific masks in conjunction with our novel end-to-end learning process outperforms several recent weakly-supervised semantic segmentation methods that use image tags only, and even some models that leverage additional supervision or training data.