Deformable Shape Completion with Graph Convolutional Autoencoders
This addresses the challenge of occlusions and missing parts in 3D scanning for applications like human body and face reconstruction, though it is incremental as it builds on existing learning-based shape completion methods.
The paper tackles the problem of completing partial 3D scans of non-rigidly deforming objects, such as human bodies and faces, by proposing a graph convolutional autoencoder method that learns a latent space for realistic shapes and optimizes to fit partial inputs, showing promising results on synthetic and real scans.
The availability of affordable and portable depth sensors has made scanning objects and people simpler than ever. However, dealing with occlusions and missing parts is still a significant challenge. The problem of reconstructing a (possibly non-rigidly moving) 3D object from a single or multiple partial scans has received increasing attention in recent years. In this work, we propose a novel learning-based method for the completion of partial shapes. Unlike the majority of existing approaches, our method focuses on objects that can undergo non-rigid deformations. The core of our method is a variational autoencoder with graph convolutional operations that learns a latent space for complete realistic shapes. At inference, we optimize to find the representation in this latent space that best fits the generated shape to the known partial input. The completed shape exhibits a realistic appearance on the unknown part. We show promising results towards the completion of synthetic and real scans of human body and face meshes exhibiting different styles of articulation and partiality.