Removing supervision in semantic segmentation with local-global matching and area balancing
It addresses the problem of reducing supervision in semantic segmentation for computer vision applications, presenting a novel approach that advances the field beyond incremental improvements.
The paper tackles unsupervised semantic segmentation by developing a model using local-global matching and area balancing, achieving state-of-the-art results of 43.6% mIoU on PascalVOC2012 and 19.4% on MS-COCO2014 without any labels.
Removing supervision in semantic segmentation is still tricky. Current approaches can deal with common categorical patterns yet resort to multi-stage architectures. We design a novel end-to-end model leveraging local-global patch matching to predict categories, good localization, area and shape of objects for semantic segmentation. The local-global matching is, in turn, compelled by optimal transport plans fulfilling area constraints nearing a solution for exact shape prediction. Our model attains state-of-the-art in Weakly Supervised Semantic Segmentation, only image-level labels, with 75% mIoU on PascalVOC2012 val set and 46% on MS-COCO2014 val set. Dropping the image-level labels and clustering self-supervised learned features to yield pseudo-multi-level labels, we obtain an unsupervised model for semantic segmentation. We also attain state-of-the-art on Unsupervised Semantic Segmentation with 43.6% mIoU on PascalVOC2012 val set and 19.4% on MS-COCO2014 val set. The code is available at https://github.com/deepplants/PC2M.