Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images
This work solves an instance confusion issue in panoptic segmentation for high-resolution image analysis, representing an incremental improvement over existing methods.
The paper tackles the problem of crop-based training in panoptic segmentation causing confusion between large object instances in high-resolution images, and proposes Intra-Batch Supervision (IBS) to address this, achieving improvements of up to +2.5 on Panoptic Quality for thing classes and up to +5.8 on pixel accuracy and precision on Cityscapes and Mapillary Vistas datasets.
Unified panoptic segmentation methods are achieving state-of-the-art results on several datasets. To achieve these results on high-resolution datasets, these methods apply crop-based training. In this work, we find that, although crop-based training is advantageous in general, it also has a harmful side-effect. Specifically, it limits the ability of unified networks to discriminate between large object instances, causing them to make predictions that are confused between multiple instances. To solve this, we propose Intra-Batch Supervision (IBS), which improves a network's ability to discriminate between instances by introducing additional supervision using multiple images from the same batch. We show that, with our IBS, we successfully address the confusion problem and consistently improve the performance of unified networks. For the high-resolution Cityscapes and Mapillary Vistas datasets, we achieve improvements of up to +2.5 on the Panoptic Quality for thing classes, and even more considerable gains of up to +5.8 on both the pixel accuracy and pixel precision, which we identify as better metrics to capture the confusion problem.