Unsupervised Total Variation Loss for Semi-supervised Deep Learning of Semantic Segmentation
This work addresses the challenge of semantic segmentation for computer vision applications when densely labeled training data is scarce, offering a semi-supervised approach that reduces annotation effort.
The paper tackles the problem of learning semantic segmentation with limited labeled data by introducing an unsupervised total variation loss that promotes piecewise smoothness in label probability images, combined with supervised loss in a semi-supervised setting, achieving significant improvements over purely supervised methods on datasets like Weizmann horse, Stanford background, and Sift Flow.
We introduce a novel unsupervised loss function for learning semantic segmentation with deep convolutional neural nets (ConvNet) when densely labeled training images are not available. More specifically, the proposed loss function penalizes the L1-norm of the gradient of the label probability vector image , i.e. total variation, produced by the ConvNet. This can be seen as a regularization term that promotes piecewise smoothness of the label probability vector image produced by the ConvNet during learning. The unsupervised loss function is combined with a supervised loss in a semi-supervised setting to learn ConvNets that can achieve high semantic segmentation accuracy even when only a tiny percentage of the pixels in the training images are labeled. We demonstrate significant improvements over the purely supervised setting in the Weizmann horse, Stanford background and Sift Flow datasets. Furthermore, we show that using the proposed piecewise smoothness constraint in the learning phase significantly outperforms post-processing results from a purely supervised approach with Markov Random Fields (MRF). Finally, we note that the framework we introduce is general and can be used to learn to label other types of structures such as curvilinear structures by modifying the unsupervised loss function accordingly.