NP-SemiSeg: When Neural Processes meet Semi-Supervised Semantic Segmentation
This work addresses uncertainty quantification in semi-supervised segmentation, which is important for safety-critical applications like medical imaging and self-driving cars, but it is incremental as it adapts an existing method to a new task.
The paper tackled the problem of improving semi-supervised semantic segmentation by addressing uncertainty in pseudo-label predictions, resulting in a new model called NP-SemiSeg that demonstrated effectiveness on benchmarks like PASCAL VOC 2012 and Cityscapes.
Semi-supervised semantic segmentation involves assigning pixel-wise labels to unlabeled images at training time. This is useful in a wide range of real-world applications where collecting pixel-wise labels is not feasible in time or cost. Current approaches to semi-supervised semantic segmentation work by predicting pseudo-labels for each pixel from a class-wise probability distribution output by a model. If the predicted probability distribution is incorrect, however, this leads to poor segmentation results, which can have knock-on consequences in safety critical systems, like medical images or self-driving cars. It is, therefore, important to understand what a model does not know, which is mainly achieved by uncertainty quantification. Recently, neural processes (NPs) have been explored in semi-supervised image classification, and they have been a computationally efficient and effective method for uncertainty quantification. In this work, we move one step forward by adapting NPs to semi-supervised semantic segmentation, resulting in a new model called NP-SemiSeg. We experimentally evaluated NP-SemiSeg on the public benchmarks PASCAL VOC 2012 and Cityscapes, with different training settings, and the results verify its effectiveness.