Detection, Pose Estimation and Segmentation for Multiple Bodies: Closing the Virtuous Circle
This work addresses a critical bottleneck in computer vision for applications like surveillance and robotics, though it is incremental as it builds on existing top-down approaches.
The paper tackles the problem of human pose estimation in multi-body proximity scenarios by proposing the BBox-Mask-Pose (BMP) method, which iteratively enforces consistency among bounding boxes, instance masks, and poses, achieving state-of-the-art results on OCHuman and COCO datasets with a 39% detection improvement in overlapping scenes.
Human pose estimation methods work well on isolated people but struggle with multiple-bodies-in-proximity scenarios. Previous work has addressed this problem by conditioning pose estimation by detected bounding boxes or keypoints, but overlooked instance masks. We propose to iteratively enforce mutual consistency of bounding boxes, instance masks, and poses. The introduced BBox-Mask-Pose (BMP) method uses three specialized models that improve each other's output in a closed loop. All models are adapted for mutual conditioning, which improves robustness in multi-body scenes. MaskPose, a new mask-conditioned pose estimation model, is the best among top-down approaches on OCHuman. BBox-Mask-Pose pushes SOTA on OCHuman dataset in all three tasks - detection, instance segmentation, and pose estimation. It also achieves SOTA performance on COCO pose estimation. The method is especially good in scenes with large instances overlap, where it improves detection by 39% over the baseline detector. With small specialized models and faster runtime, BMP is an effective alternative to large human-centered foundational models. Code and models are available on https://MiraPurkrabek.github.io/BBox-Mask-Pose.