RealDiff: Real-world 3D Shape Completion using Self-Supervised Diffusion Models
This work addresses the limited generalizability of synthetic priors for real-world 3D shape completion, offering a domain-specific solution for applications like robotics or autonomous systems.
The paper tackled the problem of 3D shape completion from partial point clouds in real-world settings, proposing RealDiff, a self-supervised diffusion model that leverages geometric cues without synthetic data, and it outperformed state-of-the-art methods in real-world completion tasks.
Point cloud completion aims to recover the complete 3D shape of an object from partial observations. While approaches relying on synthetic shape priors achieved promising results in this domain, their applicability and generalizability to real-world data are still limited. To tackle this problem, we propose a self-supervised framework, namely RealDiff, that formulates point cloud completion as a conditional generation problem directly on real-world measurements. To better deal with noisy observations without resorting to training on synthetic data, we leverage additional geometric cues. Specifically, RealDiff simulates a diffusion process at the missing object parts while conditioning the generation on the partial input to address the multimodal nature of the task. We further regularize the training by matching object silhouettes and depth maps, predicted by our method, with the externally estimated ones. Experimental results show that our method consistently outperforms state-of-the-art methods in real-world point cloud completion.