DProST: Dynamic Projective Spatial Transformer Network for 6D Pose Estimation
This addresses a fundamental computer vision problem for robotics and AR/VR applications, with incremental improvements in handling mesh-less settings.
The paper tackles 6D pose estimation from single RGB images by proposing a method based on a projective grid instead of object vertices, which improves performance by considering projective geometry. It shows that mesh-less DProST outperforms state-of-the-art mesh-based methods on LINEMOD and LINEMOD-OCCLUSION datasets and is competitive on YCBV.
Predicting the object's 6D pose from a single RGB image is a fundamental computer vision task. Generally, the distance between transformed object vertices is employed as an objective function for pose estimation methods. However, projective geometry in the camera space is not considered in those methods and causes performance degradation. In this regard, we propose a new pose estimation system based on a projective grid instead of object vertices. Our pose estimation method, dynamic projective spatial transformer network (DProST), localizes the region of interest grid on the rays in camera space and transforms the grid to object space by estimated pose. The transformed grid is used as both a sampling grid and a new criterion of the estimated pose. Additionally, because DProST does not require object vertices, our method can be used in a mesh-less setting by replacing the mesh with a reconstructed feature. Experimental results show that mesh-less DProST outperforms the state-of-the-art mesh-based methods on the LINEMOD and LINEMOD-OCCLUSION dataset, and shows competitive performance on the YCBV dataset with mesh data. The source code is available at https://github.com/parkjaewoo0611/DProST