Category-Level 3D Non-Rigid Registration from Single-View RGB Images
This addresses the challenge of registering objects with transparent or shiny surfaces that are hard to perceive with RGB-D sensors, offering a practical solution for robotics and computer vision applications.
The paper tackles the problem of 3D non-rigid registration from single-view RGB images by training a CNN to infer deformation fields, enabling reconstruction of observed models without depth data. It outperforms the Coherent Point Drift method for evaluated object categories.
In this paper, we propose a novel approach to solve the 3D non-rigid registration problem from RGB images using Convolutional Neural Networks (CNNs). Our objective is to find a deformation field (typically used for transferring knowledge between instances, e.g., grasping skills) that warps a given 3D canonical model into a novel instance observed by a single-view RGB image. This is done by training a CNN that infers a deformation field for the visible parts of the canonical model and by employing a learned shape (latent) space for inferring the deformations of the occluded parts. As result of the registration, the observed model is reconstructed. Because our method does not need depth information, it can register objects that are typically hard to perceive with RGB-D sensors, e.g. with transparent or shiny surfaces. Even without depth data, our approach outperforms the Coherent Point Drift (CPD) registration method for the evaluated object categories.