PCDreamer: Point Cloud Completion Through Multi-view Diffusion Priors
This addresses the problem of incomplete 3D shapes for applications like robotics and VR, offering an incremental improvement over existing methods by leveraging diffusion models without requiring paired data.
The paper tackles point cloud completion by using multi-view diffusion priors from large models to generate images that provide shape cues, resulting in superior performance with improved recovery of fine details.
This paper presents PCDreamer, a novel method for point cloud completion. Traditional methods typically extract features from partial point clouds to predict missing regions, but the large solution space often leads to unsatisfactory results. More recent approaches have started to use images as extra guidance, effectively improving performance, but obtaining paired data of images and partial point clouds is challenging in practice. To overcome these limitations, we harness the relatively view-consistent multi-view diffusion priors within large models, to generate novel views of the desired shape. The resulting image set encodes both global and local shape cues, which are especially beneficial for shape completion. To fully exploit the priors, we have designed a shape fusion module for producing an initial complete shape from multi-modality input (i.e.,, images and point clouds), and a follow-up shape consolidation module to obtain the final complete shape by discarding unreliable points introduced by the inconsistency from diffusion priors. Extensive experimental results demonstrate our superior performance, especially in recovering fine details.