Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation
This work addresses the challenge of making robotic platforms more flexible and generalizable in interacting with unstructured environments, representing an incremental improvement in category-level 6D pose estimation.
The paper tackled the problem of estimating 6D object poses from RGB images in a zero-shot, category-level setting, and achieved increased performance by integrating diffusion models for novel view synthesis with feature extraction and online optimization, reducing data requirements and eliminating the need for depth information.
Estimating the pose of objects through vision is essential to make robotic platforms interact with the environment. Yet, it presents many challenges, often related to the lack of flexibility and generalizability of state-of-the-art solutions. Diffusion models are a cutting-edge neural architecture transforming 2D and 3D computer vision, outlining remarkable performances in zero-shot novel-view synthesis. Such a use case is particularly intriguing for reconstructing 3D objects. However, localizing objects in unstructured environments is rather unexplored. To this end, this work presents Zero123-6D, the first work to demonstrate the utility of Diffusion Model-based novel-view-synthesizers in enhancing RGB 6D pose estimation at category-level, by integrating them with feature extraction techniques. Novel View Synthesis allows to obtain a coarse pose that is refined through an online optimization method introduced in this work to deal with intra-category geometric differences. In such a way, the outlined method shows reduction in data requirements, removal of the necessity of depth information in zero-shot category-level 6D pose estimation task, and increased performance, quantitatively demonstrated through experiments on the CO3D dataset.