High Fidelity Semantic Shape Completion for Point Clouds using Latent Optimization
This addresses the challenge of generating complete 3D shapes from partial inputs in computer vision, with incremental improvements in fidelity and method.
The paper tackles the problem of semantic shape completion for incomplete 3D point clouds by proposing a learning-based approach using generative modeling and latent optimization, achieving high fidelity reconstructions without relying on database retrieval.
Semantic shape completion is a challenging problem in 3D computer vision where the task is to generate a complete 3D shape using a partial 3D shape as input. We propose a learning-based approach to complete incomplete 3D shapes through generative modeling and latent manifold optimization. Our algorithm works directly on point clouds. We use an autoencoder and a GAN to learn a distribution of embeddings for point clouds of object classes. An input point cloud with missing regions is first encoded to a feature vector. The representations learnt by the GAN are then used to find the best latent vector on the manifold using a combined optimization that finds a vector in the manifold of plausible vectors that is close to the original input (both in the feature space and the output space of the decoder). Experiments show that our algorithm is capable of successfully reconstructing point clouds with large missing regions with very high fidelity without having to rely on exemplar based database retrieval.