Joint Calibration for Semantic Segmentation
This work addresses key challenges in semantic segmentation for computer vision applications, offering incremental improvements over existing methods.
The paper tackles the problems of overlapping multi-scale regions, class imbalance, and class competition in semantic segmentation by proposing a joint calibration method that optimizes a multi-class loss at the pixel level. It achieves state-of-the-art results with improvements of +6% in fully supervised and +10% in weakly supervised settings on the SIFT Flow dataset.
Semantic segmentation is the task of assigning a class-label to each pixel in an image. We propose a region-based semantic segmentation framework which handles both full and weak supervision, and addresses three common problems: (1) Objects occur at multiple scales and therefore we should use regions at multiple scales. However, these regions are overlapping which creates conflicting class predictions at the pixel-level. (2) Class frequencies are highly imbalanced in realistic datasets. (3) Each pixel can only be assigned to a single class, which creates competition between classes. We address all three problems with a joint calibration method which optimizes a multi-class loss defined over the final pixel-level output labeling, as opposed to simply region classification. Our method outperforms the state-of-the-art on the popular SIFT Flow [18] dataset in both the fully and weakly supervised setting by a considerably margin (+6% and +10%, respectively).