Silhouette Guided Point Cloud Reconstruction beyond Occlusion
This addresses the challenge of partial occlusion in 3D reconstruction for applications like robotics and augmented reality, representing an incremental advance.
The paper tackles the problem of reconstructing complete 3D shapes from single RGB images with occlusions by using predicted silhouettes as guidance, resulting in improved state-of-the-art performance with quantitative gains demonstrated on synthetic and real datasets.
One major challenge in 3D reconstruction is to infer the complete shape geometry from partial foreground occlusions. In this paper, we propose a method to reconstruct the complete 3D shape of an object from a single RGB image, with robustness to occlusion. Given the image and a silhouette of the visible region, our approach completes the silhouette of the occluded region and then generates a point cloud. We show improvements for reconstruction of non-occluded and partially occluded objects by providing the predicted complete silhouette as guidance. We also improve state-of-the-art for 3D shape prediction with a 2D reprojection loss from multiple synthetic views and a surface-based smoothing and refinement step. Experiments demonstrate the efficacy of our approach both quantitatively and qualitatively on synthetic and real scene datasets.