CASAPose: Class-Adaptive and Semantic-Aware Multi-Object Pose Estimation
This addresses the need for fast and memory-efficient multi-object pose estimation in applications like augmented reality and robotics, offering a novel approach that reduces the domain gap and handles occlusion.
The paper tackles the problem of joint localization and 6D pose estimation for multiple objects in RGB images, presenting CASAPose, a single-stage architecture that achieves high accuracy and efficiency by using semantic segmentation to guide keypoint recognition, outperforming state-of-the-art methods in challenging scenes with occlusion and synthetic training.
Applications in the field of augmented reality or robotics often require joint localisation and 6D pose estimation of multiple objects. However, most algorithms need one network per object class to be trained in order to provide the best results. Analysing all visible objects demands multiple inferences, which is memory and time-consuming. We present a new single-stage architecture called CASAPose that determines 2D-3D correspondences for pose estimation of multiple different objects in RGB images in one pass. It is fast and memory efficient, and achieves high accuracy for multiple objects by exploiting the output of a semantic segmentation decoder as control input to a keypoint recognition decoder via local class-adaptive normalisation. Our new differentiable regression of keypoint locations significantly contributes to a faster closing of the domain gap between real test and synthetic training data. We apply segmentation-aware convolutions and upsampling operations to increase the focus inside the object mask and to reduce mutual interference of occluding objects. For each inserted object, the network grows by only one output segmentation map and a negligible number of parameters. We outperform state-of-the-art approaches in challenging multi-object scenes with inter-object occlusion and synthetic training.