Pose2Seg: Detection Free Human Instance Segmentation
This work addresses the problem of human instance segmentation, particularly under heavy occlusion, for computer vision researchers, by introducing a novel pose-based approach and a challenging dataset, though it is incremental in focusing on a specific category.
The paper tackles human instance segmentation by proposing a pose-based framework that separates instances using human pose skeletons instead of detection bounding boxes, achieving better accuracy than state-of-the-art detection-based methods and handling occlusion more effectively. It also introduces a new benchmark dataset, OCHuman, with 8110 annotated instances and an average MaxIoU of 0.67, highlighting occlusion as a key challenge.
The standard approach to image instance segmentation is to perform the object detection first, and then segment the object from the detection bounding-box. More recently, deep learning methods like Mask R-CNN perform them jointly. However, little research takes into account the uniqueness of the "human" category, which can be well defined by the pose skeleton. Moreover, the human pose skeleton can be used to better distinguish instances with heavy occlusion than using bounding-boxes. In this paper, we present a brand new pose-based instance segmentation framework for humans which separates instances based on human pose, rather than proposal region detection. We demonstrate that our pose-based framework can achieve better accuracy than the state-of-art detection-based approach on the human instance segmentation problem, and can moreover better handle occlusion. Furthermore, there are few public datasets containing many heavily occluded humans along with comprehensive annotations, which makes this a challenging problem seldom noticed by researchers. Therefore, in this paper we introduce a new benchmark "Occluded Human (OCHuman)", which focuses on occluded humans with comprehensive annotations including bounding-box, human pose and instance masks. This dataset contains 8110 detailed annotated human instances within 4731 images. With an average 0.67 MaxIoU for each person, OCHuman is the most complex and challenging dataset related to human instance segmentation. Through this dataset, we want to emphasize occlusion as a challenging problem for researchers to study.