Prior to Segment: Foreground Cues for Weakly Annotated Classes in Partially Supervised Instance Segmentation
This work improves instance segmentation for researchers and practitioners dealing with limited mask annotations by better leveraging more abundant bounding box labels, offering a simpler architecture for this task.
This paper addresses the challenge of partially supervised instance segmentation where only a subset of classes have mask labels, while others only have bounding box labels. The authors introduce an Object Mask Prior (OMP) that helps the class-agnostic mask head learn a general concept of foreground, leading to significant improvements over the Mask R-CNN baseline and competitive performance with state-of-the-art methods on the COCO dataset.
Instance segmentation methods require large datasets with expensive and thus limited instance-level mask labels. Partially supervised instance segmentation aims to improve mask prediction with limited mask labels by utilizing the more abundant weak box labels. In this work, we show that a class agnostic mask head, commonly used in partially supervised instance segmentation, has difficulties learning a general concept of foreground for the weakly annotated classes using box supervision only. To resolve this problem we introduce an object mask prior (OMP) that provides the mask head with the general concept of foreground implicitly learned by the box classification head under the supervision of all classes. This helps the class agnostic mask head to focus on the primary object in a region of interest (RoI) and improves generalization to the weakly annotated classes. We test our approach on the COCO dataset using different splits of strongly and weakly supervised classes. Our approach significantly improves over the Mask R-CNN baseline and obtains competitive performance with the state-of-the-art, while offering a much simpler architecture.