Multi-View Object Pose Refinement With Differentiable Renderer
This work addresses object pose estimation for robotics and AR/VR applications, offering a robust and practical refinement approach that is incremental but improves upon existing methods.
The paper tackles the problem of refining 6 DoF object poses from multi-view images, particularly for methods trained on synthetic data, by introducing a differentiable renderer with an ICP-like loss function, achieving excellent performance on datasets like LineMOD and YCB-V compared to state-of-the-art methods.
This paper introduces a novel multi-view 6 DoF object pose refinement approach focusing on improving methods trained on synthetic data. It is based on the DPOD detector, which produces dense 2D-3D correspondences between the model vertices and the image pixels in each frame. We have opted for the use of multiple frames with known relative camera transformations, as it allows introduction of geometrical constraints via an interpretable ICP-like loss function. The loss function is implemented with a differentiable renderer and is optimized iteratively. We also demonstrate that a full detection and refinement pipeline, which is trained solely on synthetic data, can be used for auto-labeling real data. We perform quantitative evaluation on LineMOD, Occlusion, Homebrewed and YCB-V datasets and report excellent performance in comparison to the state-of-the-art methods trained on the synthetic and real data. We demonstrate empirically that our approach requires only a few frames and is robust to close camera locations and noise in extrinsic camera calibration, making its practical usage easier and more ubiquitous.